What will be the highest score achieved on SWE-Bench Verified in 2025?
Plus
6
Ṁ7602026
1D
1W
1M
ALL
16%
<70
31%
70-85 inclusive
53%
>85
https://openai.com/index/introducing-swe-bench-verified/
https://www.swebench.com/
Highest performance reported before 2026. Any run on https://www.swebench.com/ counts. Large AI company reported numbers count whether or not they're listed on swebench.com Other claimed scores will generally not be counted unless verified by a third party.
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Related questions
Related questions
80% on SWE-Bench Verified by Jan 1 2025
16% chance
What will be the best score on the SWE-Bench (unassisted) benchmark before 2025?
39% chance
Will any model get above human level (92%) on the Simple Bench benchmark before September 1st, 2025.
52% chance
AI resolves at least X% on SWE-bench assistance, by 2025?
Will an AI SWE model score higher than 50% on SWE-bench in 2024?
20% chance
AI resolves at least X% on SWE-bench WITH assistance, by 2028?
Will an AI score over 30% on FrontierMath Benchmark in 2025
26% chance
What will be the best score on the WebArena benchmark before 2025?
47% chance
BIG-bench accuracy 75% #2: Will SOTA for a single model on BIG-bench pass 75% by the start of 2025?
65% chance
When will SWE-bench be solved?