When will SWE-bench be solved?
Basic
4
Ṁ202030
1D
1W
1M
ALL
8%
2024
30%
2025
18%
2026
8%
2027
8%
2028
8%
2029
19%
later
Resolves to the first year in which https://www.swebench.com posts a 90% resolved score in the “verified” category. The current score is 38.80% by Amazon Q Developer Agent.
Get
1,000
and1.00
Related questions
Related questions
AI resolves at least X% on SWE-bench WITH assistance, by 2028?
What will be the best score on the SWE-Bench (unassisted) benchmark before 2025?
39% chance
AI resolves at least X% on SWE-bench assistance, by 2025?
80% on SWE-Bench Verified by Jan 1 2025
14% chance
Will an AI SWE model score higher than 50% on SWE-bench in 2024?
20% chance
AI resolves at least X% on SWE-bench without any assistance, by 2028?
Will an autonomous agent resolve 90% of tasks on SWE-bench by 2026?
70% chance
SOTA on a SWE-bench [Unassisted] in October 2024
Will an autonomous agent resolve 90% of tasks on SWE-bench by 2027?
72% chance
SOTA on a SWE-bench [Assisted] in October 2024