Why Evaluating AI Models Is Far More Complex Than Testing Traditional Software

A deep dive into Chapter 3 of 'AI Engineering' highlights that evaluating AI systems is as challenging as building them, unlike traditional software where pass-or-fail tests are straightforward. AI models can produce multiple plausible answers to the same question, making it difficult to determine which response is genuinely better. Benchmarks like GLUE, introduced in 2018, became obsolete within a year as models surpassed them, prompting the creation of harder alternatives like SuperGLUE — illustrating how evaluation standards must constantly evolve. Metrics such as perplexity, cross entropy, and bits-per-byte offer ways to measure how well a model predicts text, but these figures are only meaningful when interpreted within the context of a specific dataset. The chapter also draws a distinction between exact evaluation, suited to tasks with definitive answers like math problems, and subjective evaluation, required for assessing qualities like helpfulness or creativity.