Why Free Evaluation Servers Must Never Share Production Write Keys
A software architect describes how a production incident occurred when a free execution server replayed an evaluation job that still carried a live production write key, causing unintended writes to a shared store. The core argument is that a shadow evaluation plane must never share commit keys, queues, or compensation topics with the live serving plane, even when doing so makes code diffs easier to manage. When two planes share a key namespace, they effectively become a single failure domain regardless of how separate their dashboards appear. The author proposes a strict architectural fence where the control plane rewrites any incoming job's key before model calls begin, ensuring the free server can only write to a shadow store. The piece concludes that free-tier capacity is an availability option for canary testing, not a license to allow secondary writers to influence production state.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in