Hiring Tip: Test Engineers With a Token-Budget Ledger, Not Just a Demo

The article argues that a working demo says little about an AI engineer's skill, since the real test is how a service behaves once its budget runs out. It proposes a take-home task: build a small question-answering service over a fixed corpus that calls a hosted model, deploys to a public URL, and keeps a machine-readable ledger of every token spent. Candidates receive a fixed token allotment and must explain what the system does when that allotment is exhausted. The author provides a roughly forty-line Python sample that caps output tokens based on remaining budget and charges for prompts and completions, along with tests that check refusal at the limit. The piece also notes that free model access and hosting from MonkeyCode make the exercise practical to run at scale, while advising reviewers to verify current terms before use.