Open-source tool verifies AI predictions about real-world sound levels

A new tool called Prove It Outside challenges AI models to make falsifiable predictions about physical locations. The system requires open-weight models to commit to predictions, such as which of two places is quieter, before any verification occurs. Researchers then physically visit the locations to record sound levels using smartphones or microphones. The tool scores each model's accuracy using Brier scores only if measurements clearly distinguish between locations. This approach aims to test whether AI models' confident assertions about unseen places hold up against real-world evidence.