Retries Are Not Repeats: Five Idempotency Myths in Model-Backed Endpoints

A developer article argues that a single 200 response from a model-backed endpoint does not reveal how many times the model actually executed. It explains that client timeouts are unknown outcomes rather than failures, since the server may still be processing or finish after the client gives up. The author recommends generating an idempotency key before the first attempt, reusing it across retries, and storing completed responses with a TTL so duplicates return a conflict instead of re-running. The piece also advises honoring 429 responses with Retry-After headers or jittered backoff, and avoiding retries on 400 or 422 errors that will not succeed. It further notes that free model access reduces token costs but not duplicate side effects, so retry budgets should be measured in time and effects. A standard-library Python lab is included to count real executions and test key-based deduplication.