Context information

SWE-bench: evaluating patches against real software issues

SWE-bench evaluates systems on software issues collected from GitHub: a system receives a repository and an issue, then produces a candidate patch. The project supplies datasets and a Docker-based evaluation harness. This makes repository-level task completion a distinct evaluation target from isolated code-generation examples.

External source ↗

Provenance: SWE-bench project documentation
Updated: 2026-11-08T22:15:34.951965Z