Lilian Weng's blog post breaks down AI scaling laws and their real-world limits

AI researcher Lilian Weng published a detailed analysis titled 'Scaling Laws, Carefully' on her blog Lil'Log in June 2026, examining how model size, data volume, and compute collectively follow power-law relationships in large language model training. The post revisits the long-standing debate between the Kaplan scaling approach, which prioritized model size over data, and the Chinchilla findings, which showed that model parameters and training tokens should scale proportionally. Weng explains that the Chinchilla model, though four times smaller than DeepMind's Gopher, outperformed it by training on four times more tokens with the same compute budget. The post also addresses data-constrained scenarios, warning that repeatedly training on the same data yields diminishing returns and causes overfitting, especially in larger models. Weng cautions that scaling laws are empirical tools, not physical laws, and that small errors in curve-fitting can lead to vastly wrong predictions when extrapolating to expensive large-scale training runs.