Alphabell.
Paper · Inference efficiency

ADRS

Barbarians at the Gate: How AI is Upending Systems ResearchThe paper argues that systems research suits AI-driven discovery because performance problems come with reliable verifiers (simulators and workloads), names the approach AI-Driven Research for Systems (ADRS), and runs the open-source OpenEvolve framework on case studies that include Mixture-of-Experts inference load balancing and LLM inference over SQL queries.

The loop

LLMs inside OpenEvolve (Gemini 2.5 Flash and Flash Lite for the MoE case) propose and mutate code for the expert-parallelism load balancer that places MoE expert replicas on GPUs during LLM inference, and a 168-line PyTorch simulator scores each candidate on load imbalance and rebalancing runtime, with the best programs seeding the next generation. A second case evolves the row and field reordering that maximizes prefix KV-cache reuse when SQL queries call an LLM on every row. The output is faster serving code for LLMs, the same class of model that wrote it, which is the indirect loop.

The loop
ADRS
Paper · arXiv
  1. LLMs propose and mutate load balancer code
  2. Best programs seed the next generation
  3. Output is faster serving code for LLMs
  1. LLMs propose and mutate load balancer code
  2. Best programs seed the next generation
  3. Output is faster serving code for LLMs
↻ The improved system does the next round, and the loop turns again.

Why it is a road to recursion

Because systems problems come with cheap, reliable verifiers, the same evolve-and-measure loop can be pointed at the serving stack of the LLMs that do the evolving, so each inference-efficiency gain makes the next round of search cheaper.

Evidence

For MoE expert-parallelism load balancing, the evolved algorithm matched the baselines' imbalance factor while cutting rebalancing runtime to 3.7 ms, a 5.0x speedup over a non-public frontier-lab reference implementation (19.6 ms), in a run of about five hours that cost under $10.

systems-researchopenevolvemixture-of-expertsllm-servingkv-cachealgorithm-discovery

Related loops

More inference efficiency →
PaperarXiv
Context Language Models
  1. Model updates context file
  2. Skill-optimization evolves instructions
  3. Model uses improved instructions
  4. Model manages context better
  5. repeat
Inference efficiencyRu-Lin Shao et al. · 2026
BenchmarkarXiv
FlashInfer-Bench
  1. LLM agents write GPU kernels
  2. Faster kernels make LLM inference cheaper
  3. New serving traces define next kernel tasks
  4. repeat
Inference efficiencyUW, CMU, NVIDIA, UC Berkeley (Xing et al.) · 2026
PaperarXiv
Glia
  1. LLM agents write inference cluster code
  2. Designs lower latency and GPU cost
  3. Lowers cost of serving LLMs running Glia
  4. repeat
Inference efficiencyHamadanian et al. (MIT CSAIL) · 2025