The hidden pitfalls of per-token LLM billing that developers learn the hard way

·1 views

A developer running a multi-provider LLM gateway in production has shared hard-won lessons about the complexities of per-token billing across providers like OpenAI, Anthropic, Google, and DeepSeek. Key challenges include inconsistent pricing per request, scattered usage data in streaming responses, and silent client aborts that can result in untracked costs. The author also highlights the importance of checking account balances before a request is made and settling charges only after completion. Currency handling across providers adds another layer of complexity to accurate metering. These insights come from building the billing layer for kral.ai, a managed LibreChat platform designed for enterprise use.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Tutorial Guide Shows How to Build Cross-Platform CLI and GUI Tools Using Tcl/Tk

A new tutorial published on cgicoffee.com walks developers through building cross-platform command-line and graphical user interface tools using Tcl/Tk. The guide targets developers looking to create applications that run consistently across different operating systems. Tcl/Tk is a long-standing scripting language and GUI toolkit known for its portability and simplicity. The article was shared on Hacker News in April 2026, drawing minimal engagement at the time of reporting.

0 comments Read more at Hacker News

ProgrammingDEV Community ·

Developer finds six tests that passed without actually testing anything

A developer discovered six separate instances in one project where automated tests reported success without genuinely verifying the properties they were designed to check. Examples included a database restore script that confirmed an empty database's schema as valid, a health-check running at high volume instead of the low-volume condition it was meant to simulate, and a bucketing test whose assertion and input were the same hardcoded constant. Even after identifying the pattern and writing a rule to prevent it, the developer immediately wrote another hollow test, highlighting that awareness alone does not prevent the mistake. The author concludes that catching such tests requires a mechanical discipline: for every guard, explicitly identify what input would make it pass without the target property holding, and confirm that input appears as a failing negative case in the test suite.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

More Context Can Make AI Answers Worse, Not Better, Research Shows

AI models advertise large context windows of up to one million tokens, but their effective working memory is significantly smaller in practice. As more text is added to a prompt, the model's attention — which is fixed and must be divided across all tokens — becomes diluted, reducing answer quality. Research, including the 'Lost in the Middle' study by Liu et al., found that information placed in the middle of long prompts is retrieved far less accurately than content near the beginning or end. Real-world data compounds the problem, as multiple similar or conflicting document versions make it harder for the model to identify the correct source. Experts recommend sending fewer but higher-quality chunks — typically three to five — and placing key instructions at the top and bottom of a prompt to improve reliability.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Dev ships Three.js iOS game in 17 days after four silent, hard-to-debug failures

A developer successfully published a Three.js game to the iOS App Store in 17 days across 247 commits, running inside a WKWebView via Capacitor with no network calls. Four critical issues nearly derailed the project, each failing silently without errors or logs, making each one resemble a different problem entirely. Offscreen rendering ran at 2.5 minutes per frame due to Chrome defaulting to a software rasterizer; switching to the Metal GPU backend brought performance to 38–60 fps with a single flag change. On iOS, WebKit silently kills the GPU process without firing the standard webglcontextlost event, requiring additional detectors — including a draw-call floor check — to reliably catch and recover from context loss. A StoreKit timing issue and a world-tiling modulo mismatch were among the other silent failures that demanded careful workarounds before the app could ship.

0 comments Read more at DEV Community