How Retry Storms Turned a Provider Outage into 200 Failed Calls in n8n
The author of a build-in-public series on production LLM workflows in n8n describes a morning when a provider went down and their workflow retried the same failed call about two hundred times. The retries consumed quota, flooded logs with identical errors, and delayed other queued jobs because the loop held executions open. The author argues that a retry policy involves three separate decisions: wait time between attempts, number of attempts, and when to stop entirely. The proposed remedies are exponential backoff with a five-minute cap, honoring a provider's Retry-After header, and adding full jitter to prevent synchronized retries from overwhelming a recovering provider. The article also introduces circuit breakers, which halt calls to a provider after consecutive failures.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in