By when will AI score >= 80% on FrontierMath
➕
Plus
28
Ṁ5184
2050
0.3%
2024
8%
2025
20%
2026
43%
2027
19%
2028
2%
2029
5%
2030
0.6%
2035
0.5%
2050

https://epochai.org/frontiermath

pass@1 no-holds barred other than training on the test set

Abstract:

We introduce FrontierMath, a benchmark of hundreds of original, exceptionally challenging mathematics problems crafted and vetted by expert mathematicians. The questions cover most major branches of modern mathematics -- from computationally intensive problems in number theory and real analysis to abstract questions in algebraic geometry and category theory. Solving a typical problem requires multiple hours of effort from a researcher in the relevant branch of mathematics, and for the upper end questions, multiple days. FrontierMath uses new, unpublished problems and automated verification to reliably evaluate models while minimizing risk of data contamination. Current state-of-the-art AI models solve under 2% of problems, revealing a vast gap between AI capabilities and the prowess of the mathematical community. As AI systems advance toward expert-level mathematical abilities, FrontierMath offers a rigorous testbed that quantifies their progress.

Get
Ṁ1,000
and
S3.00
Sort by:
opened a Ṁ1,000 2030 YES at 1.0% order

Title could be improved.

@jim "On which year will AI first score >= 80% on FrontierMath"

@jim I admit this suggestion isn't perfect since some of your options like "2035" actually refer to a range of years (e.g. "2035" refers to the years 2031 through 2035).

So, how about changing the title to "During which period will AI score >= 80% on FrontierMath?" or just "When will AI score >= 80% on FrontierMath?" and editing the options "2035" and "2050" to "2031 - 2035" and "later than 2035".

© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules