New CLI tool detects bibliographic data leakage in materials science ML models

A new open-source command-line tool called materials-confounding-check (mcc) helps researchers identify whether machine learning models trained on materials science datasets are exploiting bibliographic metadata — such as author, journal, or publication year — rather than genuine chemical structure. This phenomenon, known as bibliographic confounding or metadata leakage, was documented as widespread across five real materials science tasks in a paper by Jablonka et al. (2026). The tool runs four falsification sub-tests on a dataset, assessing whether bibliography can be predicted from chemical descriptors, whether a metadata-only model rivals a full-descriptor model, and whether performance collapses under author- or time-based data splits. Statistical robustness is ensured through a null distribution of 100 permutations and a 95th-percentile decision threshold, avoiding manually tuned margins. The project, authored by Pedro Sordo Martínez and licensed under AGPL-3.0, is available on GitHub and targets a gap left by generic leakage tools that do not address materials-specific bibliographic confounding.