Event
Release of the FrontierCS 2 computer science research and development benchmark
1 reports · 1 sources
Researchers introduced the FrontierCS 2 benchmark, comprising 48 open-ended tasks. Their approach was to take a paper and the code released by its authors, delete the paper's contribution, and use the remaining code as a human reference baseline. Unlike earlier benchmarks that only give a problem description and score a program after it runs, FrontierCS 2 places agents in a research environment, gives them an inherited runnable repository, and lets them submit patches at any time; the evaluator returns only a development-suite score, while the evaluator, the held-out final suite, and the references remain private. The tasks span 13 domains, from machine learning systems and high-performance computing to databases, security, and robotics. In addition to these 48 tasks, FrontierCS 2 also holds more than 100 private tasks, to be announced in the future. The researchers evaluated 12 agents, and the best performer was Claude Opus 5.5, which passed 58% of runs.
Report timeline
- FrontierCS 2: A Benchmark for Computer Science R&D Released
Researchers introduce FrontierCS 2, a benchmark of 48 open-ended tasks. Their approach takes a paper and the code its authors released, deletes the paper's contribution, and uses the remaining code as the human reference baseline. Unlike earlier benchmarks that only give a problem description and grade one program after the run, FrontierCS 2 places agents inside a research environment where they inherit a runnable repository and can submit patches at any time. The evaluator returns only a development-suite score, while the evaluator, the held-out final suite, and the reference remain private.
HuggingFace Daily Papers ·