MIT researchers say GPT-6 Astra fixed a small Qwen model by editing 0.016% of its weights directly

Four researchers at MIT CSAIL, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, published a blog post on November 2, 2026 describing what they call Weight-Space Programming: asking a large language model, here GPT-6 Astra, to read and modify another neural network's parameters directly, by reasoning and running Python analysis rather than by training it with an optimizer. The authors describe the findings as very early.

The Qwen experiment

The headline result concerns Qwen3.5-0.8B, a small open model that often got stuck in reasoning loops on math problems: 93% of its MATH-500 answers hit the 32,768-token limit. Working in a Python environment without a GPU, and using only practice problems from the GSM8K and MATH training sets, Astra traced the looping to one direction written by the MLP blocks of layers 15 and 19. It subtracted that direction from 64 output columns in each layer, a total of 131,072 weights, or 0.016% of the model.

With a budget of 50 forward passes, the authors report MATH-500 accuracy rising from 7.9% to 48.1%, GSM8K from 13.8% to 67.1% and the IFEval instruction-following benchmark from 21.0% to 62.3%. The share of MATH-500 answers hitting the token limit fell to 33.1%. With 10 or 25 passes the gains were much smaller. On Gemma 3 270M, which rarely loops, no edit moved MATH-500 by more than about a point. The authors conclude that such edits can clear an obstacle hiding an ability but cannot add one the model lacks.

The post also covers smaller networks: Astra reportedly identified which of four games a policy had been trained on from its weights alone, repaired a stalled PPO car-on-ramp policy by changing 3.5% of its weights, and wrote MinAtar game-playing networks from scratch that beat PPO trained for 100 million steps.

Caveats and criticism

None of these results has been independently replicated, and all figures come from the authors. In a blog post published on November 3, Sean Goedecke argues that the work shows impressive mental arithmetic rather than superhuman research. He says the game networks can be read with a simple per-cell weight heatmap, and that removing a looping direction resembles existing steering or abliteration techniques a skilled human could apply. He also notes that Astra had Python for the Qwen task but not for the toy networks, and maintains that normal training remains the better way to change a model.

The authors say they are building a benchmark and harness to evaluate such methods more consistently. Claims about uses such as auditing models or merging them are presented as possibilities, not demonstrated results.