AI Models Can Memorise and Leak Verbatim Fragments of Training Data

Large AI models do not simply learn general patterns from training data — they can memorise and reproduce identifiable fragments of it verbatim, according to a decade of documented research. Two main attack methods exploit this: membership inference, which determines whether a specific record was included in a training dataset, and data extraction, which recovers memorised content word-for-word. Memorisation tends to increase with model scale and dataset size, meaning the industry's push for larger models and bigger datasets has amplified rather than reduced the risk. Membership inference alone poses serious privacy concerns, as confirming a person's record was in a training set can reveal sensitive information — such as medical history — even without exposing the record's contents. These vulnerabilities underpin ongoing legal disputes over AI-reproduced copyrighted content and privacy research showing that personal data can leak from production AI systems.