Supervising Agents That Outlive Their Processes: A Controller Loop in Elixir OTP
The author runs coding agents in Docker containers on a kanban board, with each task managed by a TaskRunner GenServer in an Elixir/OTP application. Deploys replace the app container and kill these processes, but the agent containers can keep running, leaving the database showing sessions as running when they are not. To recover, a reconciler GenServer periodically looks for orphaned sessions and tries to re-adopt them rather than marking them failed. Re-adoption relies on a wrapper script that writes output to a log and the exit code to a marker file, which the reconciler polls for. That design had a flaw: if a container is rescheduled to another node, its marker file disappears, leaving the poll loop waiting forever.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)
Log in to join the discussion and vote.
Log in