Study Links Pretraining Quality Directly to Reinforcement Learning Gains in LLMs

Researchers used chess as a controlled sandbox to study how pretraining choices affect the success of reinforcement learning (RL) in large language models. The study tracked models ranging from 5 million to 1 billion parameters through a pipeline of pretraining, supervised fine-tuning, and RL on chess puzzles. Key findings showed that post-RL performance is highly predictable from pretraining loss, and that RL reward gains scale approximately linearly with the number of pretraining tokens. The researchers also found that RL does more than refine existing behavior — on harder problems, it surfaces reasoning paths that were nearly absent after fine-tuning alone. Validation on a math-domain model confirmed the same patterns, suggesting that robust pretraining is a prerequisite for effective RL, not merely a starting point.