AI benchmark reveals formatting quirks can skew model performance rankings

A Kaggle benchmarking study evaluated AI models on a fictional security assessment task using synthetic data. The initial pilot showed Gemini 3.7 Flash scoring below an earlier model, but removing a single Markdown formatting fence reversed their ranking without altering the models' actual answers. The experiment measured models' ability to analyze technical evidence and written authority separately across 48 synthetic cases. Researchers noted that exact citation matching and formatting compliance significantly impacted scores, which should be interpreted alongside measurement limitations. The follow-up expanded to 72 cases with more complex authority schemes while maintaining synthetic data.