This market matches Cybersecurity: Cybench from the AI 2025 Forecasting Survey by AI Digest.

The best percentage of tasks solved by an AI system on Cybench's original 40 tasks as of December 31st 2025.

Resolution criteria

This resolution will use AI Digest as its source.

Which AI systems count?

Any AI system counts if it operates within realistic deployment constraints and doesn't have unfair advantages over human baseliners.

Tool assistance, scaffolding, and any other inference-time elicitation techniques are permitted as long as:

• There is no systematic unfair advantage over the humans described in the Human Performance section (e.g. AI systems are allowed to have multiple outputs autograded while humans aren't, or AI systems have access to the internet when humans don't).

• Having the AI system complete the task does not use more compute than could be purchased with the wages needed to pay a human to complete the same task to the same level

The PASS@k elicitation technique (which automatically grades and chooses the best out of k outputs from a model) is a common example that we accept up to k=10 on this benchmark because professional human teams at CTF competitions are generally allowed to attempt multiple submissions (often with cooldown periods in between) but they are not typically allowed unlimited attempts and sometimes there are points penalties. Since these policies vary by competition we made the judgement call to limit AI systems to at most k=10, and beleive this doesn't give AI systems a clear unfair advantage over human teams.

If there is evidence of training contamination leading to substantially increased performance, scores will be accordingly adjusted or disqualified.