Descriptions:
SWE-Marathon is a benchmark for long-horizon autonomous software work: 20 project-scale tasks spanning product clones, library rewrites, and ML engineering. We discuss what happens when coding agents run for tens to hundreds of millions of tokens, why full-stack evals need computer-use verifiers, and why reward-hacking resistance is now central to benchmark design.
Speakers:
– Rishi Desai (Abundant AI): Rishi Desai is an ML Engineer at Abundant AI, where he works on RL environments and SWE benchmarks for coding agents.
X/Twitter: https://x.com/rishi_desai2
LinkedIn: https://www.linkedin.com/in/rishi-desai1/
GitHub: https://github.com/RishiDesai
