Five few-shot examples nearly doubled the cost of a short LLM prompt
A developer found that adding five worked examples to a short extraction prompt almost doubled its per-call cost. Because APIs bill for every input token on every request, the examples are resent and charged each time, even though the model keeps no memory between calls. In the author's test, the examples made up about half of the input cost, and the multiplier stayed roughly the same across several major models. The effect matters most for short tasks, since long documents dilute the examples' share of the bill. The article suggests using structured output for format issues and keeping only the examples that address genuine judgment calls.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in