BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software

Abstract

A challenging, realistic benchmark of diverse real-world open-source software for evaluating LLM agents on automatically compiling projects, plus OSS-Build-Agent, a strong baseline with an enhanced build-instruction retrieval module.