Fastest AI inference API varies by region, 30-day latency study finds

·15 views

A 30-day latency study of AI inference APIs found that no single provider is fastest everywhere, as results shift with the client's location. Fireworks AI had the quickest network response from US Central and Tokyo, while OpenRouter led from São Paulo. For time to first token, Groq's gpt-oss-120b model was fastest, with median times of 0.8 to 1.5 seconds across the three regions, and Google's Gemini Flash-Lite model was slower with higher tail latency. The researchers noted that SambaNova's network response differed 15-fold between Tokyo and São Paulo, and that closest proximity did not guarantee the fastest first token. Token-generation throughput and first-token times for Cerebras and SambaNova were not measured, and German probe results were excluded due to a throttling issue.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Reviving an Abandoned Fastify Plugin for RFC 9457 Problem Details

A developer revisited a Fastify plugin, fastify-rfc9457, that they had published and then left untouched for about a year. Its existing tests still passed, but the lint script had never worked and its dependency had moved to a new major version, so they updated it and released version 1.0.1 through a more secure GitHub Actions publishing flow. The plugin addresses the common problem of APIs returning inconsistent error formats, which forces clients to write custom parsers. It implements RFC 9457, the IETF standard for HTTP error responses that defines a consistent shape using the application/problem+json content type. The plugin applies this format automatically across routes, including validation failures and not-found errors, so developers don't have to format every error by hand.}

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Governance Gaps: Actions Outside the Tool Registry and Who Owns Them

A events table received rows from two paths: an AI assistant calling a registered tool, and a plain HTTP endpoint that bypasses the tool registry and the AI side-effect ledger. The author argues that governance should be judged by which execution paths were verified, not by how many tools are registered. The runtime registers 61 tools, but external MCP tools appear in a separate admin catalogue requiring higher permissions, leaving auditors with an incomplete view. The gate treats unregistered tools as a matter for the executor, so passing the gate does not confirm registration. The author also cites an earlier mistake in which a hand-maintained metadata table drifted from the live tool set, and recommends checking the registry against actual entry points.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Runs Personal AI Agent on Home NAS Instead of Cloud

A developer built a personal AI agent that runs in a container on a home UGREEN DXP2800 NAS rather than a cloud service. The agent's identity is defined by plain-text files covering its conduct rules, personality, the owner's profile, and daily notes, which it reads before responding. Each Monday it pulls tasks from Notion, generates an HTML recap, and emails it, with safeguards against duplicate sends. The agent also wrote and deployed a public static demo page, agent-mailbox, that shows only fictional data, while real data stays on the home machine. The project was submitted to a Hacktoberfest open-source AI challenge.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Networking Series Concludes With Focus on Active Directory and Penetration Testing Basics

A DEV Community post marks the finale of a 40-episode networking masterclass, framing it as foundational training for security work. The article argues that understanding protocols, ports, and traffic flow is essential for offensive security researchers and penetration testers working in enterprise environments. It compares this groundwork to anatomy before surgery, stressing that systems cannot be secured without understanding how network communication behaves. The piece celebrates the completion of the series while tying its lessons to Active Directory and pentesting fundamentals.

0 comments Read more at DEV Community