Alphabell.
Foundation · Self-reward and self-play

AlphaZero

AlphaZero, introduced in "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm", generalises the AlphaGo Zero approach into one algorithm that learns chess, shogi and Go from random play, given only the game rules. A neural network that guides tree search is trained only on games the system plays against itself.

The loop

The current network, combined with tree search, plays games against itself, and those games become the training data that teach the same network to predict its own search-improved moves and the game winner. The updated network then plays the next self-play games, so each version of the model produces the training signal for the next, with no human game data.

The loop
AlphaZero
Foundation · arXiv
  1. Network plays games against itself
  2. Games become the training data
  3. Same network trains on its own moves
  4. Updated network plays next self-play games
  1. Network plays games against itself
  2. Games become the training data
  3. Same network trains on its own moves
  4. Updated network plays next self-play games
↻ The improved system does the next round, and the loop turns again.

Why it is a road to recursion

It is the clearest working case of a model generating its own training signal and surpassing human-designed systems without human data, the template later self-play and self-training methods for language models follow.

Evidence

Starting from random play and given no domain knowledge except the game rules, AlphaZero reached superhuman play in chess, shogi and Go within 24 hours and defeated a world-champion program in each.

self-playreinforcement-learningtree-searchgamesdeepmind

Related loops

More self-reward and self-play →
PaperarXiv
Self-Rewarding LMs
  1. Model generates candidate responses to prompts
  2. Model scores them itself as a judge
  3. Scores become preference pairs for DPO
  4. Trained model becomes next generator and judge
  5. repeat
Self-reward and self-playMeta, NYU (Yuan et al.) · 2024
PaperarXiv
Absolute Zero
  1. Model proposes code reasoning tasks
  2. Model solves them for accuracy reward
  3. Python executor verifies the answers
  4. Curriculum shifts as solver improves
  5. repeat
Self-reward and self-playTsinghua, BIGAI, Penn State (Zhao et al.) · 2025
PaperarXiv
rStar-Math
  1. Policy model runs Monte Carlo Tree Search
  2. Verified trajectories train next policy and PPM
  3. PPM guides search to produce better data
  4. Self-evolution improves both models
  5. repeat
Self-reward and self-playMicrosoft Research Asia · 2025