Loop index
Projects, papers, benchmarks, datasets and mini-projects where AI is part of the loop that improves AI. Each entry states its loop and how far it has turned.
From the lab
Model Observatory
Replay measured Flint and alphaBell Qwen Open Q8_0 captures, then compare the published Nanbeige and Qwen decoder configurations.
The conditional 32K K/V view calculates raw f16 full-attention tensor size if a runtime enables caching. Nanbeige's released config sets use_cache: false.
The DeepSeek Flash 1017 release lens compares 11 paired publisher-reported scores and checks the two releases' published configuration files. Matching configs do not explain the reported gains.
The Ling 3.1 Flash 1019 file audit compares pinned release files: eight setup and loader files match byte for byte, while all 54 weight-shard LFS hashes differ. File hashes do not explain the reported benchmark gains.
From the lab
MiniMax Video Budget
See how sampled frames become two-frame groups and vision placeholders in the pinned MiniMax M3.1 Flash processor. A shareable calculator and JSON export make the arithmetic easy to inspect.
The count covers vision placeholders at processed frame dimensions. It does not measure total prompt tokens, memory, speed or video quality.
From the lab
MOSS Blind Layout Check
Bring one target layout and two locally generated MOSS images. Hide Base and SFT labels while you transcribe visible text and mark its observed box, then reveal pair-specific results.
The browser exports manual annotations and image hashes without image bytes. This is one user-supplied pair, not OCR or a general model ranking.
From the lab
Agent Request Evidence Ladder
Read three source-linked agent traces as separate steps: reasoning, a tool request, the observed tool result and the intended delivery. Find the boundary between an attempted request and a completed action.
Read the companion guide for the source trail and a way to apply the same evidence check to other traces.
From the lab
Kernel Trap Cards
A candidate passes one fixed input. Choose the second test that exposes a missing tail, wrong stride, overflow, skipped row or aliased buffer. The browser runs each candidate and reference and shows the first mismatch.
Five original CPU teaching examples inspired by published GPU-kernel correctness research. No model call or upload.

- Gemini models propose code changes
- Evaluators keep best scoring versions
- Finds better Gemini training kernels
- Lowers cost to train next Gemini models
- repeat

- Coding agent edits its own code
- Scored child agents enter the archive
- Archived agents make later self modifications
- repeat

- RL agent lays out TPU blocks
- Better layouts make AI hardware cheaper
- Agent pre-trains on earlier chip generations
- repeat

- Coding agent edits LLM training script
- Runs a 5-minute training job
- Keeps change if validation improves
- Changes transferred to larger models
- repeat

- Research agent writes harness mutation blueprints
- Coding agent implements harness module changes
- Winning harness becomes the new champion
- New champion harness runs further generations
- repeat

- Aligned model generates training data
- Reward model judges and filters it
- Data trains next intermediate instruct model
- Next model generates better alignment data
- repeat

- Model generates candidate responses to prompts
- Model scores them itself as a judge
- Scores become preference pairs for DPO
- Trained model becomes next generator and judge
- repeat

- Claude Code writes its own code changes
- Changes become next Claude Code versions
- Release becomes harness for next development round
- repeat

- LLM agents write and benchmark kernels
- Kernels speed up AI model training
- Session data post-trains smaller kernel models
- repeat

- Coding agent writes fine-tuning code
- Agent post-trains a base LLM
- Trained model scored on target benchmark
- One turn of model training another
- repeat

- Learned optimizers update other learned optimizers
- Training keeps best performing optimizer parameters
- Better optimizers train each other faster
- repeat

- LLM reads reasoning model thoughts
- LLM flags reward hacking behavior
- Signal feeds back into training reward
- Trains more aligned capable agents
- repeat

- Frontier AI agents
- implement AI training pipelines
- and cluster orchestration systems
- to build their successors
- repeat

- Qwen3 generates tasks and code
- executes code for answer keys
- trains on verified keys
- corrects its own false beliefs
- repeat

- Coding agents design serving systems
- Systems are verified and deployed
- Serving systems run the agents
- Agents operate with higher efficiency
- repeat

- Coding agents write infrastructure patches
- Patches improve training and inference systems
- Improved systems train better foundation models
- Better models power stronger coding agents
- repeat

- Student model fails a reasoning step
- Student writes a critique of the error
- Critique conditions a frozen teacher model
- Teacher distills improved logic back into student
- repeat

- Agent solves open scientific problems
- Agent modifies its own harness
- Kernel verifies modifications via checks
- Agent uses new harness for research
- repeat

- SplatLab reproduces 3DGS baselines
- proposes and codes extensions
- scores candidates via Elo tournament
- uses feedback to refine research outputs
- repeat

- Meta-agent analyzes past training runs
- Evolves optimizer agent policy and code
- Optimizer agent manages new training runs
- Improves training of subsequent models
- repeat

- Agent proposes scaffold edits
- Agent scores edits before running
- Agent selects highest scoring edit
- Agent uses improved scaffold
- repeat

- Teacher models evolve tools and skills
- Policy trains with adapted harness
- Policy generates new trajectories
- Trajectories guide further harness adaptation
- repeat

- Research agents refine target system
- Modify training data and harness
- Evaluate against validation feedback
- Submit improved final agent
- repeat

- Policy analyzes its own failures
- Proposes new harness candidates
- Policy trains via RL on new harness
- Successor policy tackles harder tasks
- repeat

- Cortex tests harness changes
- evaluates new skills and rules
- retains only net-gain modifications
- updates the agent's harness
- repeat

- Frozen model attempts exact QA tasks
- Model authors new DSL operators
- Independent implementations verify operators via consensus
- Verified operators expand model's executable library
- repeat

- Proposer agent builds environments
- Solver generates rollouts
- Rollouts guide next environments
- Solver trains on new environments
- repeat

- Coder generates patches
- Reviewer grades patches to guide search
- Coder output helps reviewer refine rubric
- Reviewer provides better guidance to coder
- repeat

- Agent evolver automates workflow design
- Scores candidate skills on target agents
- Improves its own evolution skill
- Produces better target agents
- repeat

- Model searches for multi-agent workflows
- Model selects the best executions
- Model trains on selected executions
- Model improves its base capabilities
- repeat

- Simulator co-evolves tasks and adversaries
- Generates challenging web agent scenarios
- Agent trains on resulting curriculum
- Agent handles harder tasks and attacks
- repeat

- Agent edits inference engine code
- Engine is tested for throughput
- Agent inherits experiences for next session
- Faster engine serves the agent
- repeat

- UMM generates images and programs
- Execution verifies programs against specifications
- Verified renders and programs form training data
- Data trains UMM visual and text capabilities
- repeat

- Model updates context file
- Skill-optimization evolves instructions
- Model uses improved instructions
- Model manages context better
- repeat

- Outer agent rewrites inner agent code
- Accepted rewrites become the incumbent agent
- Discovered agent becomes the outer loop agent
- repeat

- AI system writes accelerator hardware
- Qwen runs on the new accelerator
- Qwen finds optimizations for the accelerator
- Optimizations improve next accelerator generation
- repeat

- Coding agent edits LLM training code
- Agent benchmarks code on TPU hardware
- Verdict becomes context for next hypothesis
- Wins raise training throughput for same models
- repeat

- LLM optimizers critique deep research reports
- Optimizers rewrite research agent prompts
- Optimized system runs the next queries
- repeat

- Qwen3-14B edits GPT training script
- Edit is trained and scored
- Score rewards Qwen3-14B weights update
- Trained checkpoint re-run in autoresearch loop
- repeat

- LLM agents write GPU kernels
- Faster kernels make LLM inference cheaper
- New serving traces define next kernel tasks
- repeat

- AI systems would design chips
- Chips would train more capable AI
- That AI designs the next chips
- repeat

- LLM agents write inference cluster code
- Designs lower latency and GPU cost
- Lowers cost of serving LLMs running Glia
- repeat

- LLMs propose and mutate load balancer code
- Best programs seed the next generation
- Output is faster serving code for LLMs
- repeat

- Auditor model tests target model
- LLM judge scores target behavior
- Feeds into next Claude model release
- repeat

- LLM agents propose hypotheses on AI tasks
- Agents implement and test them
- Findings Memory steers later proposals
- Validated finding directly improves AI system
- repeat

- Agent synthesizes agentic pretraining data
- Pipeline adjusts training set in real time
- Data feeds back into model training
- repeat

- LLM ensemble rewrites candidate programs
- Evaluator scores and selects best programs
- Evolved mixture-of-experts load-balancing loss
- Loss improves LLM training
- repeat

- Reflection LLM writes revised module prompts
- Winning prompt candidates are kept
- Optimized prompts return to same system
- New system traces feed next reflection round
- repeat

- Agent rewrites LLM training script
- Script makes LLM training faster
- Benchmark scores the recovered speedup
- Speeds up agent's own model training
- repeat

- Designer agents propose language model architectures
- Verifier agents pre-train and evaluate designs
- Verified results feed evolutionary population
- Language models search for better architectures
- repeat

- Language model generates a self-edit
- Edit is applied as weight update
- RL rewards self-edit by downstream performance
- repeat

- Frontier LLMs write CUDA kernels
- Fastest variants seed each new round
- Output feeds next kernel-writing model
- repeat

- QwQ-32B writes CUDA kernels
- Model refines kernels using compiler feedback
- RL rewards model on measured speedup
- repeat

- Model proposes code reasoning tasks
- Model solves them for accuracy reward
- Python executor verifies the answers
- Curriculum shifts as solver improves
- repeat

- KernelLLM writes Triton GPU kernels
- Faster kernels make AI training cheaper
- Verified kernels train next kernel model
- repeat

- o3-mini injects bugs and writes issues
- SWE-agent solves tasks for expert trajectories
- Trajectories fine-tune Qwen 2.5 Coder
- Open coding agent trained on agent data
- repeat

- Best agent changes its own code
- Edited agent is benchmarked and archived
- Archived agent makes the next edit
- repeat

- LLM agents propose ML research ideas
- Agents run experiments with tree search
- Agents write up results and papers
- Output is new knowledge about models
- repeat

- Agent replicates published ML research
- LLM judge grades the replication
- Measures ability to do ML research
- Research produces better future models
- repeat

- Policy model runs Monte Carlo Tree Search
- Verified trajectories train next policy and PPM
- PPM guides search to produce better data
- Self-evolution improves both models
- repeat

- LLM writes replacement GPU kernel
- Kernel is timed against baseline
- Workloads are neural network operators
- Makes agent's own training faster
- repeat

- Agents do AI research tasks
- Scored against human expert baselines
- Measures automation of AI research
- Research improves future AI systems
- repeat

- Agents build and train ML models
- Submissions scored against human thresholds
- Measures open-ended ML research ability
- Agents improve their own training code
- repeat

- Meta agent programs new agent designs
- Designs are evaluated on target tasks
- Results are added to an archive
- Meta agent uses archive for next design
- repeat

- LLM builds worse answers for preference pairs
- Judge samples reasoning traces and verdicts
- Judge is fine-tuned on correct verdicts
- Improved judge labels next iteration's data
- repeat

- CriticGPT critiques ChatGPT code answers
- Humans catch errors using critiques
- Improves labels for RLHF training
- Labels train the next ChatGPT
- repeat

- GPT-4 proposes preference-optimization losses
- Candidate losses fine-tune language models
- Evaluation scores feed back into prompt
- Best loss aligns other LLMs
- repeat

- Llama-3-70B scores pages for educational value
- Regressor trained on labels filters corpus
- Filtered corpus pretrains new language models
- LLM judgment sets next training diet
- repeat

- Agent runs interpretability experiments
- Agent finds spurious background cues
- Selection retrains classifier final layer
- Improves robustness of interpreted model
- repeat

- GPT-4 rewrites the improver program
- Each rewrite is scored by meta-utility
- Best rewrite becomes next round improver
- repeat

- Aider writes new code for Aider releases
- New code becomes the next release
- Next release builds the following version
- repeat

- Language model writes evaluation questions
- Evaluations test other language models
- Exposes behaviors worsened by RLHF
- Evaluates next round of training
- repeat

- Model critiques and revises its own responses
- Model is fine-tuned on the revisions
- Model judges responses to train preference model
- Preference model provides RL reward signal
- repeat

- Language model writes step-by-step rationales
- Correct rationales are kept as dataset
- Model is fine-tuned on self-generated set
- Improved model generates next rationale dataset
- repeat

- Network plays games against itself
- Games become the training data
- Same network trains on its own moves
- Updated network plays next self-play games
- repeat

- RNN controller writes child network architecture
- Child validation accuracy returns as reward
- Updates make controller propose better architectures
- repeat

- Proof searcher tests proof techniques
- Executes useful rewrites of its own code
- Rewritten machine searches on its new code
- repeat
Missing a loop?
Send us a project, paper or repo with the loop in one sentence and we will review it for the index.