corpus-scrub 0.1.0 launches as open-source tool to strip PII and secrets from LLM training data

A developer has released corpus-scrub 0.1.0, a local-first command-line tool designed to detect and redact personally identifiable information and secrets from machine learning training corpora before model training begins. The tool scans JSONL, TXT, and Parquet files, identifying sensitive data such as emails, phone numbers, IBANs, credit card numbers, and API keys using a combination of Microsoft Presidio, spaCy NER models, and regex rules ported from the gitleaks project. All processing runs locally with no data sent to external services, addressing privacy risks highlighted in recent research showing that LLMs can memorize and leak training data. The tool supports English in its current MVP, with multilingual name detection for Spanish, German, and French via dedicated spaCy large models, and offers three redaction modes: masking, hashing, or dropping. Compliance motivations include GDPR and Article 10 of the EU AI Act, which restrict the use of unstructured personal data in training sets without consent.