Why Postgres, not the GenServer, should hold the agent task queue
The author noticed a Camelot dashboard showing dispatched tasks that conflicted with other views, and traced the mismatch to a UI bug that had been fixed. Agent runs are costly in both tokens and money, so concurrency must be capped globally and per user. Rather than refusing work when capacity is full, the pool accepts every request and returns the caller's position in line, leaving callers free of retry logic. Because a queue held only in GenServer memory would vanish on each of the project's frequent deploys, the durable queue lives in Postgres as queued session rows. The GenServer keeps only a derived in-memory index, which a reconciler rebuilds from the database on boot.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in