I made a version of this market which allows for closed source LLMs: https://manifold.markets/RyanGreenblatt/by-when-will-85-be-reached-on-the-p

This is your chance to win free mana betting against SG, which is a guaranteed winning strategy exploited by top traders such as jackson

Now James is on the action. Thanks!

Note that this prize doesn't allow for close source models to be used in doing the actual task.

Of course, distillation is possible etc.

Did you mean “by 2025” or “in 2025”? Meaning, how would it resolve it the prize were claimed in 2024?

You're right, thanks. Correcting.

@mckiev i might take you up on the offer, but what's your reasoning for 85% accuracy? we're at 30% right now

Vibes based for the most part ;)

Tasks seem easy, 100s of billions being invested in ai right now, and there is hype and status around beating this benchmark