Dev team documents four real bugs from seven months running a self-hosted AI stack

A development team has been self-hosting its own AI stack on local hardware since January 2026, accumulating over 2,100 commits under an MIT license. Four documented bugs from the public changelog reveal practical lessons: a Cloudflare 100 MB upload cap silently rejected large files before they reached the API, requiring a chunked upload protocol to fix. A static model catalog failed to update when the GPU host swapped models, prompting a shift to dynamic discovery via LiteLLM's model info endpoint. Rate limits from external AI providers caused silent failures during parallel research tasks, addressed in version 1.40.0 with per-provider semaphores, exponential backoff, and explicit error surfacing instead of quiet local fallbacks. A fourth issue involved context window overflow, where even a 262K-token window proved insufficient in some cases, leading to structured truncation and error handling rather than silent degradation.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in