TensorSharp adds Vulkan backend support via GGML integration

·1 views

TensorSharp, an open-source local LLM inference engine, has released an initial Vulkan backend by integrating the external GGML project, with a native Vulkan implementation planned for a later release. The update was driven by strong community demand for Vulkan support. The developer has verified functionality on an Nvidia GeForce RTX 3080 Laptop GPU and Intel UHD Graphics on Windows, but AMD GPU compatibility remains untested. Benchmarks comparing TensorSharp against llama.cpp on the Vulkan backend show competitive performance across decode throughput, prefill throughput, and time-to-first-token metrics. The project supports a wide range of models and is fully compatible with OpenAI and Ollama APIs, running on Windows, macOS, and Linux.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

DeepSeek Leads on Price and Speed as Chinese AI APIs Face Head-to-Head Test

A software architect with a decade of production system experience conducted weeks of stress testing on four major Chinese AI APIs — DeepSeek, Qwen, Kimi, and GLM — evaluating p99 latency, failover behavior, and cost per million tokens under real workloads. DeepSeek's V4 Flash model emerged as the top price-performance pick, delivering around 60 tokens per second with stable tail latency at just $0.25 per million output tokens. Qwen, backed by Alibaba, stood out for offering the widest range of model sizes, including budget options as cheap as $0.01 per million tokens. Kimi commanded the highest prices at $3.00–$3.50 per million tokens but justified the cost with superior performance on reasoning-heavy tasks. GLM, developed by Zhipu AI, distinguished itself with the strongest Chinese-language quality and multimodal support not matched by the other three providers.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Residential Proxies Explained: Choosing the Right IP Strategy for Developers

Developers building scrapers or automation tools often face blocks and CAPTCHAs in production, making proxy selection a critical technical decision. Residential proxies use real ISP-assigned IPs, making them harder for anti-bot systems to detect compared to datacenter alternatives. Different proxy types serve distinct purposes: rotating residential IPs suit large-scale scraping, while static ISP proxies work better for stateful tasks like logins and account sessions. Providers such as Oxylabs and Bright Data cater to enterprise-scale needs, while IPRoyal, DataImpulse, and Webshare are recommended for budget or prototype use cases. Most providers offer a standard host-port endpoint with username-password authentication, allowing developers to control IP rotation or session stickiness directly through the username string.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Physics Graduate Mark Tony Begins Coding Journey with Payilagam

Mark Tony, a fresh entrant to the technology field with an undergraduate background in Physics, has announced the start of his coding journey. He describes the move as fulfilling a passion he was unable to pursue during his college years. Tony has begun his learning venture in collaboration with Payilagam, a coding education platform. He has chosen the DEV Community as the starting point for documenting and sharing his developer journey.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Ungoverned AI Workloads Are Quietly Inflating Enterprise Cloud Bills

As organizations rapidly adopt AI services, costs from LLM endpoints, GPU instances, and vector databases are escalating without adequate oversight. Engineers are deploying AI workloads across cloud environments, but few teams have set budget limits, alerts, or usage guardrails. Some environments have seen AI service costs double within 30 days, with surprises only surfacing at month-end billing. Industry observers are framing this as a FinOps and cloud security challenge rather than a purely technical AI problem. Experts argue that the same governance disciplines applied to traditional cloud infrastructure must now be extended to AI workloads before costs spiral further.

0 comments Read more at DEV Community