Alphabell.
Paper · Self-modifying agents

Language Models Can Rewrite Their Own Weights

A frontier model demonstrates the ability to reason about its own failures and directly rewrite its own weights to improve its accuracy on held-out tests.

Language Models Can Rewrite Their Own Weights
Figure from the paper, arXiv. Source
The loop

A 27B language model diagnoses its own failures and directly edits its own weights without gradients or training. These edited weights are then used by the model to achieve higher accuracy on subsequent unseen problems.

  1. Model diagnoses its own failure
  2. Model computes weight edits
  3. Model applies edits to itself
  4. Model answers new questions better
  1. Model diagnoses its own failure
  2. Model computes weight edits
  3. Model applies edits to itself
  4. Model answers new questions better
↻ The improved system does the next round, and the loop turns again.

Why it is a road to recursion

It demonstrates that a model can improve its own fundamental parameters through reasoning rather than gradient descent, creating a direct path to recursive self-improvement.

Evidence

The model's own edit makes it 6.6 points more accurate on 490 unseen problems.

weight editingself-correctioninference scaling