How 178 simultaneous trip report uploads stress-tested a solo developer's LLM pipeline

A traveller imported 178 trip reports at once from a two-year Canada-to-Chile journey, overwhelming a small travel site that previously held only 9 reports. Each published report triggers three background jobs — text moderation, place extraction, and image moderation — all of which call an AI model via a queue. The system's worker processes ten jobs per minute to stay within the model's rate limit, meaning the full burst took roughly 35 minutes to clear, delaying any other users publishing during that window. The next morning, around 98 photos were flagged for human review because the image moderation model had repeatedly failed to respond, exhausting all three retry attempts. The developer traced each failure through stored status reasons and run logs, distinguishing genuine errors from expected system behaviour such as cron timeouts that only appeared problematic on the surface.