Queue Depth-Based Router Proposed to Stop Secret Prompts Leaking to Cloud
A workflow proposal suggests that AI coding agents on shared workstations need a router that classifies prompts before deciding where to send them. The problem arises when local GPU queues become congested, prompting developers to redirect all requests — including those containing sensitive data like API keys — to remote hosted endpoints. The proposed router uses four signals — secret classification, local queue depth, offline status, and a wait-time budget — to determine whether a prompt should be processed locally or remotely. Only prompts classified as non-secret are permitted to leave the machine, and only when local waiting would exceed a defined time threshold. The design prioritises explainability and auditability, with each routing decision stored as serialisable JSON so incident reviews can trace exactly why a prompt was sent off-device.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in