Alphabell.
Research and evals

Research automation

AI that proposes hypotheses, runs experiments and writes up machine learning research.

Mini-projectGitHub
autoresearch
  1. Coding agent edits LLM training script
  2. Runs a 5-minute training job
  3. Keeps change if validation improves
  4. Changes transferred to larger models
  5. repeat
Research automationAndrej Karpathy · 2026
Mini-projectGitHub
autoresearch-distillation
  1. Qwen3-14B edits GPT training script
  2. Edit is trained and scored
  3. Score rewards Qwen3-14B weights update
  4. Trained checkpoint re-run in autoresearch loop
  5. repeat
Research automationExperiential Labs (Naihin, Fallah) · 2026
PaperarXiv
DeepScientist
  1. LLM agents propose hypotheses on AI tasks
  2. Agents implement and test them
  3. Findings Memory steers later proposals
  4. Validated finding directly improves AI system
  5. repeat
Research automationWeng et al. (Westlake University) · 2025
ProjectGitHub
The AI Scientist-v2
  1. LLM agents propose ML research ideas
  2. Agents run experiments with tree search
  3. Agents write up results and papers
  4. Output is new knowledge about models
  5. repeat
Research automationYamada et al. (Sakana AI) · 2025

Other loop types

Self-modifying agentsTools and harnessTraining dataSelf-reward and self-playKernels and compilersHardware and chipsInference efficiencyArchitecture and optimizer searchAI R&D evalsInterpretability and oversight