Alphabell.
Oversight

Interpretability and oversight

AI used to explain, audit and supervise other AI, with the findings fed back into training.

PaperarXiv
CoT monitoring
  1. LLM reads reasoning model thoughts
  2. LLM flags reward hacking behavior
  3. Signal feeds back into training reward
  4. Trains more aligned capable agents
  5. repeat
Interpretability and oversightOpenAI (Baker et al.) · 2025
ProjectAnthropic
Petri
  1. Auditor model tests target model
  2. LLM judge scores target behavior
  3. Feeds into next Claude model release
  4. repeat
Interpretability and oversightAnthropic · 2025
PaperarXiv
CriticGPT
  1. CriticGPT critiques ChatGPT code answers
  2. Humans catch errors using critiques
  3. Improves labels for RLHF training
  4. Labels train the next ChatGPT
  5. repeat
Interpretability and oversightOpenAI (McAleese et al.) · 2024
PaperarXiv
MAIA
  1. Agent runs interpretability experiments
  2. Agent finds spurious background cues
  3. Selection retrains classifier final layer
  4. Improves robustness of interpreted model
  5. repeat
Interpretability and oversightMIT CSAIL (Rott Shaham et al.) · 2024
PaperarXiv
Model-written evals
  1. Language model writes evaluation questions
  2. Evaluations test other language models
  3. Exposes behaviors worsened by RLHF
  4. Evaluates next round of training
  5. repeat
Interpretability and oversightAnthropic (Perez et al.) · 2022

Other loop types

Self-modifying agentsTools and harnessTraining dataSelf-reward and self-playKernels and compilersHardware and chipsInference efficiencyArchitecture and optimizer searchResearch automationAI R&D evals