InfraDev: Can Coding Agents Develop AI Infrastructure, and How Can We Tell?
The loop: Coding agents develop production-grade AI infrastructure like Megatron-LM and vLLM within the InfraDev benchmark. This improves the evaluation of infrastructure engineering, guiding the development of agents that can build and optimize their own training and inference systems.
A benchmark evaluating coding agents on real-world pull requests to AI infrastructure repositories like Megatron-LM and vLLM.







