How to Handle LLM API Failures with Retry Logic, Backoff, and Circuit Breakers

LLM APIs fail in four main ways: rate limits (429), provider overload, timeouts, and degraded output that still returns a 200 status. Engineers should retry freely on transient errors like 429s, 5xx responses, and read-only timeouts, but must never retry validation errors or authentication failures. Exponential backoff combined with random jitter prevents retry stampedes, and the provider's Retry-After header should always override any client-side wait formula. A total deadline cap ensures retries do not run indefinitely when a provider is severely degraded. Wrapping all of this in a circuit breaker stops a struggling provider from cascading failures into the broader system.