Temperature 0 Does Not Guarantee Deterministic LLM Output, Developer Finds

A developer's CI test failed despite identical prompts and temperature set to 0, prompting an investigation into whether that setting guarantees repeatable output. The write-up argues temperature 0 is only a request for greedy decoding, with the endpoint deciding how strictly to honor it, and that floating-point behavior, batching, software versions, and silent model swaps can all cause variation. It also explains that max_tokens limits tokens rather than characters, that finish_reason and token counts must be checked, and that stop-sequence handling differs across providers. The piece concludes that a model name is only a label and the endpoint is the real contract, and it provides a short repeat-run probe script for testing an endpoint's consistency.