RepairBench

Leaderboard of frontier models for program repair.

If you use RepairBench, please cite:

@techreport{repairbench,
  title={RepairBench: Leaderboard of Frontier Models for Program Repair}, 
  author={André Silva and Martin Monperrus},
  year={2024},
  url={https://arxiv.org/abs/2409.18952}, 
  number = {2409.18952},
  institution = {arXiv},
}

For the code to reproduce the benchmark, please refer to https://github.com/ASSERT-KTH/repairbench-framework

Structure

• results includes all prompts, patches, and evaluation results
• scripts contains scripts used to parse results into other formats
• website contains the leaderboard's website code