MIT Researchers Use an LLM to Read and Edit Neural Network Weights Directly

MIT CSAIL researchers Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab published a November 2026 study called "Sparks of Weight-Space Programming" on having a general-purpose language model work directly with neural network parameters. In one case, the model repaired a repetition-loop problem in the small open model Qwen3.5-0.8B by modifying only about 0.016% of its weights, with no gradient descent or training data. The edit lifted scores on MATH-500, GSM8K, and IFEval substantially. The authors also tested whether the model could identify what games four small policies were trained to play by reading their raw weights, and it correctly named all four. The work challenges the assumption that weights can only be written by optimizers, though the authors report failure cases alongside their successes.