Alphabell.
Paper · Training data

AutoMaster

AutoMaster: Autonomous Post-Training by Making the Same Model a Better TeacherAn autonomous post-training system uses an LLM agent to develop interventions that improve a teacher model's rollouts, yielding better fine-tuning data for target models.

AutoMaster
Figure from the paper, arXiv. Source
The loop

An LLM agent analyzes teacher trajectories to design skills that suppress undesirable habits and elicit effective behaviors. This improves the teacher model, which then generates higher quality data to fine-tune a better successor.

The loop
AutoMaster
Paper · arXiv
  1. Agent analyzes teacher trajectories
  2. Agent designs skills to suppress bad habits
  3. Teacher model generates improved rollouts
  4. Target model fine-tunes on better data
  1. Agent analyzes teacher trajectories
  2. Agent designs skills to suppress bad habits
  3. Teacher model generates improved rollouts
  4. Target model fine-tunes on better data
↻ The improved system does the next round, and the loop turns again.

Why it is a road to recursion

Automating the design of post-training interventions allows models to continuously refine their own teaching behaviors without human-engineered skill definitions.

Evidence

Improves mean performance over Vanilla SFT on successful unguided teacher trajectories by 1.42 to 8.99 percentage points across 4B and 9B target models.

post-trainingdata-generationagentic-workflow