Doubling API concurrency cut load time 14% but introduced call failures, dev reverts change

·1 views

A developer building Holder Atlas, a token-holder geography tool, found that generating a single world map required roughly 110 sequential Nansen API calls averaging nearly two seconds each, resulting in a cold load time of around 40 seconds. To speed this up, they doubled the concurrency pool from 4 to 8 workers and raised the rate limit from 5 to 10 requests per second during a benchmark test on four tokens in September 2026. The wider configuration improved the median cold load time by only 14 percent, from 40.3 seconds to 34.5 seconds, while the 95th-percentile time barely changed. More critically, the higher concurrency produced three failed API calls that the narrower setup had avoided, causing one token's cached result to differ from its cold-run output. The developer reverted to the original 4-wide, 5 rps defaults, concluding that per-call latency — not worker count — is the true bottleneck.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

AI Tool Retrieval Fails 95% of the Time When Users Paraphrase Requests

A structured evaluation of deferred AI tool loading found that retrieval works nearly perfectly when users mirror the exact vocabulary in tool descriptions, but collapses to around 5% recall when they paraphrase naturally. The experiment tested BM25 retrieval across 100 synthetic enterprise tools at three description detail levels — terse, realistic, and verbose — using 200 tasks split equally between vocabulary-matching and paraphrase queries. Verbose descriptions, which cost roughly double the tokens of realistic ones, showed no meaningful improvement in paraphrase recall, exposing that adding more words drawn from the same vocabulary provides no extra retrieval surface. Widening the shortlist from 5 to 10 tools only pushed paraphrase recall from 5% to 10%, doubling context costs for minimal gain. The core finding is that a vocabulary gap between how tools are described and how users actually speak creates a near-invisible failure mode, where the model receives a plausible but wrong shortlist and silently improvises.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How LINQ's Select Operator Cuts Unnecessary Data Load in .NET Apps

A common performance issue in .NET applications involves Entity Framework loading all columns of a database entity even when only a few are needed by the UI. Using LINQ's Select operator, developers can project only the required columns, reducing data transfer, memory usage, and query execution time. For reusable projections, C# Data Transfer Objects (DTOs) — especially using the concise record syntax introduced in C# 9 — are recommended over anonymous types. Select can also handle computed fields, nested related entities, and conditional logic that translates to SQL CASE WHEN statements. The SelectMany operator further extends this by flattening collections across related entities into a single efficient query.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

STON.fi Omniston Widget Offers Standalone TON Connect Mode for Lightweight dApps

STON.fi's Omniston Widget supports a standalone TON Connect configuration that lets developers embed a swap widget with built-in wallet connectivity, without setting up a separate TON Connect instance. In this mode, the developer provides only a TON Connect manifest URL and a DOM mount point, while the widget handles all internal wallet connection logic. The setup suits landing pages, dashboards, and lightweight dApps where the swap widget is the primary feature requiring wallet access. However, standalone mode is not appropriate for applications that already manage their own TON Connect instance; those should use the integrated mode instead. Developers must host a publicly accessible TON Connect manifest file on the same domain as their application, following TON Connect requirements for HTTPS, open CORS policy, and raster image formats for the app icon.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why PostgreSQL with pgvector beats dedicated vector databases for most AI apps

A growing number of engineers argue that specialized vector databases like Pinecone are unnecessary for the majority of AI applications, with PostgreSQL's pgvector extension offering a capable alternative. The debate gained prominence during the 2023–2024 generative AI boom, when startups raised hundreds of millions to build dedicated vector search engines on the assumption that relational databases could not keep pace. Critics of dedicated vector databases highlight four key risks: dual data sources, broken ACID transactional consistency, added network latency, and a fragmented security model. By contrast, storing vectors directly in PostgreSQL via pgvector allows atomic transactions, simpler queries, and zero additional infrastructure cost for datasets in the tens of thousands. Proponents suggest that only applications requiring billions of vectors or highly specialized ANN performance genuinely justify the operational complexity of a standalone vector database.

0 comments Read more at DEV Community