How a Dev Team Cut AI Testing Costs by Swapping Out Only the Model

Engineers building a course product on Zephyr Cloud's AI Platform faced high costs and slow test cycles because every chat interaction required real AI model calls. To solve this, they used Playwright to drive the actual desktop app end to end, but replaced only the model-facing functions with scripted responses. The harness intercepted the three key functions — message routing, specialist replies, and the evaluator — while leaving all other app logic, UI rendering, and data persistence fully intact. This approach meant tests reflected the genuine user experience without triggering real model calls or accumulating API costs. The method allowed the team to run the full suite repeatedly without the suite growing slower or more expensive over time.