Prompt caching cut one agent's prompt costs by 73% without changing its rules

·5 views

A developer describes a small recipe agent whose prompt was reworked in four steps while the model, rules, and wording stayed the same. The original design placed a constantly changing date at the top, so the cache wrote every prompt but never read one back, making the test evening costlier than having no cache. Moving the date to the user message and placing per-person details after the shared instructions let later messages reuse most of their prompt. The author also found that the tool list, which differed by subscription plan, blocked cache reuse, and fixed it by sending the same list to everyone and enforcing plan limits separately. Across the evening, the prompts ended up 73% cheaper, and the whole test cost less than a cent.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Krea 2 cleanup in ComfyUI: one shared recipe across three input and output setups

Three Krea 2 cleanup workflows in ComfyUI follow the same skeleton: reverse-prompting the input image, rebuilding it through three depth-structure passes with one shared LoRA, then upscaling with 4x-UltraSharp. The variants differ only in I/O, covering a single image, a local folder loop, and a RunningHub ZIP batch. All three use the same Krea 2 Turbo model, text encoder and VAE, with 8-step KSampler passes and a console that selects render stages, the final long edge, and the reverse-prompt route. The single-image graph is presented as the clearest example, and the author suggests a 2048-pixel long edge as the best value for cost, though this is guidance rather than a measured benchmark. The source also notes that the in-graph label on the batch variant is outdated, so the node graph should be trusted over the note text.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Per-customer budget caps for Claude API calls, with prompt caching priced correctly

A developer guide explains why Anthropic's console limits don't suit products serving many customers, since they cannot isolate one heavy account. It proposes a per-customer monthly spending cap for the Messages API, implemented in TypeScript with Redis. The key cost pitfall is that the usage field's input_tokens excludes cached reads and cache writes, which must be priced separately or costs are undercounted. The approach includes a price table per model, a cost function that splits 5-minute and 1-hour cache writes, and throws on unknown models so new ones are not silently priced at zero. The article notes that batch discounts, long-context tiers and server-tool fees are not modeled, so the resulting cap is a floor rather than a ceiling.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Tracking LLM spend per customer in Node.js using AsyncLocalStorage

A common problem is that an OpenAI invoice shows total spending but not which customer generated it, which matters for flat-priced plans. The article proposes attaching a customer ID to the request's async context once, through middleware, instead of threading it through every function. Developers can wrap the OpenAI client so each call reads that context and records its cost, with feature tags added via nested context scopes. Background jobs can start their own context tied to the relevant customer. Costs are converted to dollars at write time using a dated price table, with attention to cached input tokens billed at a lower rate.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

fold() vs reduce() in Kotlin: How the Starting Accumulator Shapes Android Code

The article contrasts Kotlin's fold() and reduce(), which both collapse a collection into a single value but differ in how they begin. fold() takes an explicit seed, so its result type can differ from the element type, while reduce() starts from the first element and requires a compatible type. The author illustrates this with a download-sync example that sums byte counts both ways and builds a transfer report from a list of parts. The piece also notes that reduce() suits merging same-shaped values, such as combining time windows into one covering window. It highlights failure modes like empty collections causing exceptions and overflow in byte totals, all tied to the choice of seed.

0 comments Read more at DEV Community