Alphabell.
Paper · Self-reward and self-play

When Can Self-Play Fix a False Belief? Answer Keys, Pass Bands and Replacement Rules in an Execution-Grounded Loop

An analysis of execution grounded self play reveals that training on self generated code execution only fixes a model's false beliefs if the answer key comes from an unplanted teacher model rather than the model's own flawed implementation.

When Can Self-Play Fix a False Belief? Answer Keys, Pass Bands and Replacement Rules in an Execution-Grounded Loop
Figure 1 of the paper, arXiv. Source
The loop

Qwen3 4B Base generates output prediction tasks and Python code, then executes the code to derive answer keys. The model trains on these self generated, execution verified keys to correct its own planted false beliefs about task specifications.

The loop
When Can Self-Play Fix a False Belief? Answer Keys, Pass Bands and Replacement Rules in an Execution-Grounded Loop
Paper · arXiv
  1. Qwen3 generates tasks and code
  2. executes code for answer keys
  3. trains on verified keys
  4. corrects its own false beliefs
  1. Qwen3 generates tasks and code
  2. executes code for answer keys
  3. trains on verified keys
  4. corrects its own false beliefs
↻ The improved system does the next round, and the loop turns again.

Why it is a road to recursion

It demonstrates that execution grounded self play can correct a model's internal false beliefs, provided the verification signal escapes the model's own flawed implementation.

Evidence

A correct key from an unplanted teacher raised the truth rate from 0.03 to 0.58 after three rounds.

self playcode executionbelief correctionexpert iteration

Related loops

More self-reward and self-play →
Self-Rewarding LMs
PaperarXiv
Self-Rewarding LMs
  1. Model generates candidate responses to prompts
  2. Model scores them itself as a judge
  3. Scores become preference pairs for DPO
  4. Trained model becomes next generator and judge
  5. repeat
Self-reward and self-playMeta, NYU (Yuan et al.) · 2024
How Much Does the Hint Matter? Scope Conditions for Hindsight Self-Distillation in Financial Reasoning
PaperarXiv
How Much Does the Hint Matter? Scope Conditions for Hindsight Self-Distillation in Financial Reasoning
  1. Student model fails a reasoning step
  2. Student writes a critique of the error
  3. Critique conditions a frozen teacher model
  4. Teacher distills improved logic back into student
  5. repeat
Self-reward and self-playJohnny Yu and Cyril Gorlla · 2026
The Red Queen Gödel Machine
PaperarXiv
The Red Queen Gödel Machine
  1. Coder generates patches
  2. Reviewer grades patches to guide search
  3. Coder output helps reviewer refine rubric
  4. Reviewer provides better guidance to coder
  5. repeat
Self-reward and self-playAlex Iacob et al. · 2026