Alphabell.
Benchmark · Inference efficiency

ServerBench

ServerBench: How can your coding agents design specialized LLM serving systems?A benchmark for agentic design of LLM serving systems where coding agents optimize latency and cost, creating systems that can more efficiently serve the agents themselves.

ServerBench
Figure from the paper, arXiv. Source
The loop

Coding agents design and optimize specialized LLM serving systems, which then run the agents themselves with higher efficiency and lower cost.

The loop
ServerBench
Benchmark · arXiv
  1. Coding agents design serving systems
  2. Systems are verified and deployed
  3. Serving systems run the agents
  4. Agents operate with higher efficiency
  1. Coding agents design serving systems
  2. Systems are verified and deployed
  3. Serving systems run the agents
  4. Agents operate with higher efficiency
↻ The improved system does the next round, and the loop turns again.

Why it is a road to recursion

Automating the optimization of LLM serving infrastructure creates a direct path to cheaper and faster agentic execution.

Evidence

Claude Sonnet 5.5 achieves 2.97 dollars per dollar on Gemma E4B, 4.87x the vLLM reference

coding-agentsllm-servingbenchmarks

Related loops

More inference efficiency →
SEIS
PaperarXiv
SEIS
  1. Agent edits inference engine code
  2. Engine is tested for throughput
  3. Agent inherits experiences for next session
  4. Faster engine serves the agent
  5. repeat
Inference efficiencyZhen Xu et al. · 2026
Context Language Models
PaperarXiv
Context Language Models
  1. Model updates context file
  2. Skill-optimization evolves instructions
  3. Model uses improved instructions
  4. Model manages context better
  5. repeat
Inference efficiencyRu-Lin Shao et al. · 2026
FlashInfer-Bench
BenchmarkarXiv
FlashInfer-Bench
  1. LLM agents write GPU kernels
  2. Faster kernels make LLM inference cheaper
  3. New serving traces define next kernel tasks
  4. repeat
Inference efficiencyUW, CMU, NVIDIA, UC Berkeley (Xing et al.) · 2026