Mid-stream failover made my chat API answer the same prompt twice — switch models before the first token or not at all

A client's transcript came in looking like a slip of the tongue from a language model: three cut-off sentences about caching, then the exact same question answered again, in a slightly different voice, spliced together as one continuous message. The culprit was failover logic in my chat router. I run an OpenAI-compatible /v1/chat/completions endpoint with 50+ models behind one integration (https://x402.freeq.one/tools/llm_chat.html), and the eco tier picks the cheapest healthy provider for each request. One night an upstream started handing out 429s — but only after accepting the connection an