Intel's Dokimi Brings A/B Testing to AI Agent Context

Dokimi, an Intel project, is a tool for testing how different context inputs affect an AI model's performance on specific tasks. The method treats elements such as system prompts, documentation, user profiles, and agent skills as variables in a controlled experiment. A standardized set of questions is run through a baseline configuration and enhanced variants, and a separate LLM judge scores the answers on dimensions like accuracy, completeness, and conciseness. The framework then computes performance differences and tests their statistical significance to determine whether gains are real or just noise. The project is designed to replace subjective prompt tuning with data-driven decisions about which context to ship.