llama.cpp Drops Key Multi-GPU Flag, But New Features Restore Lost Speed

·1 views

A developer running a dual Tesla P40 GPU setup built their entire inference pipeline around the llama.cpp flag '-sm row', which delivered 12–14 tokens/sec compared to roughly 7 for layer split mode. In July 2026, upstream llama.cpp removed the flag entirely — not as a deprecation but a full deletion — forcing a shift to layer split as the only multi-GPU option. Earlier testing had also revealed that an alternative 'graph' split mode, claimed to be 40% faster, caused hard CUDA crashes on Pascal-generation GPUs, underscoring the risk of relying on unverified community benchmarks. A separate architecture-level bug in Gemma 4's shared KV layers had already made row split unusable for that model family, revealing that split mode affects correctness, not just performance. However, parallel inference slots and speculative decoding on the new stack pushed aggregate throughput beyond what the removed flag had previously achieved, ultimately leaving the setup faster than before.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Ten Infrastructure Controls Every AI Agent Needs Before Going Live

A developer's customer-support AI agent got stuck in an overnight retry loop with a CRM tool, generating a $4,200 OpenAI bill without completing any useful work due to the absence of a circuit breaker. The incident highlights a broader pattern: roughly 95% of enterprise generative-AI pilots in 2025 yielded no measurable return, with failures tied to missing operational safeguards rather than model quality. Unlike read-only chatbots, deployed agents write to databases, issue refunds, and call APIs, making unchecked failures far more consequential. Key recommended controls include hard retry caps, deny-by-default tool allowlists, human confirmation gates for irreversible actions, and infrastructure-enforced cost ceilings — all implemented in code outside the model's reach. Research cited in the article found that action-level privilege enforcement reduced AI agent attack success rates from 70.3% to 7.3%, underscoring that safety infrastructure, not prompt engineering, is the critical line of defence.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Four-Part Framework to Keep Coding Agents Focused and Under Control

A practical guide published on DEV Community outlines how developers can write better prompts for AI coding agents by treating them as formal change specifications rather than casual requests. The recommended framework structures prompts into four components: Task, Context, Scope, and Acceptance Criteria. Defining scope explicitly — such as restricting edits to specific folders or prohibiting new dependencies — helps prevent agents from making unintended changes to unrelated code. Acceptance criteria should be observable and verifiable, with agents directed to discover validation commands from project configuration rather than assuming common defaults. The approach aims to produce smaller, reviewable diffs while keeping AI-generated changes aligned with existing project conventions.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How Agent-to-Agent Marketplaces Work in 2026: A Developer's Technical Overview

Agent-to-agent (A2A) marketplaces are decentralized directories that allow autonomous AI agents to discover, negotiate with, and pay for capabilities offered by other agents, functioning differently from traditional API gateways. The dominant 2026 implementation follows the x402 specification, a lightweight HTTP-based protocol enabling metered, pay-per-use interactions between stateful agent programs. Each agent publishes a machine-readable JSON-LD descriptor stored on IPFS, detailing its inputs, outputs, pricing in USDC, and service-level agreements. Payments are settled on-chain or via layer-2 rollups without a custodial intermediary, using signed payment proofs verified by smart contracts on networks like Base or Optimism. The marketplace architecture comprises loosely coupled components — including a registry, discovery API, payment verifier, optional router, and agent SDK — linked only by a shared descriptor schema and the x402 payment format.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Screenshots Plus Brief Notes Help AI Coding Assistants Fix UI Issues Faster

Providing an AI coding assistant with a screenshot alongside a short written note gives it the visual context needed to make accurate interface edits without repeated prompting. A screenshot reveals layout, spacing, color contrast, and element relationships that are difficult to convey through text descriptions alone. However, screenshots have limitations — they capture only a single moment and cannot show interactive behaviors, hover states, or how a page responds across different screen sizes. A brief written note compensates for these gaps by clarifying which elements need changing, which should remain untouched, and what the intended outcome is. Together, the two inputs reduce back-and-forth and help the assistant make the correct edit on the first or second attempt.

0 comments Read more at DEV Community