Will Matthew and Tamay win their bet on longer AI timelines by 2026?
19
1kṀ7755
resolved Apr 1
Resolved
NO
From https://www.lesswrong.com/posts/X3p8mxE5dHYDZNxCm/a-concrete-bet-offer-to-those-with-short-ai-timelines @MatthewBarnett and Tamay believe that AI timelines are longer than expected. Specifically, they were offering "1:1 odds that no more than one of the following events will occur by 2026-01-01: - A model/ensemble of models achieves >80% on all tasks in the MMLU benchmark - A credible estimate reveals that an AI lab deployed EITHER >10^30 FLOPs OR hardware that would cost $1bn if purchased through competitive cloud computing vendors at the time on a training run to develop a single ML model (excluding autonomous driving efforts) - A model/ensemble of models will achieve >90% on the MATH dataset using a no-calculator rule - A model/ensemble of models achieves >80% top-1 strict accuracy on competition-level problems on the APPS benchmark - A gold medal for the IMO Grand Challenge (conditional on it being clear that the questions were not in the training set) - A robot that can, from beginning to end, reliably wash dishes, take them out of an ordinary dishwasher and stack them into a cabinet, without breaking any dishes, and at a comparable speed to humans (<120% the average time) - Tesla’s full-self-driving capability makes fewer than one major mistake per 100,000 miles" Resolves YES if they win their bet (no more than one of the above occurs by the end of 2025).
AI
Technical AI Timelines
Get
Ṁ1,000
to start trading!

🏅 Top traders

#NameTotal profit
1Ṁ212
2Ṁ59
3Ṁ37
4Ṁ19
5Ṁ17
Sort by:

@Austin Both this and the 2030 version resolve No. Matthew and Tamay conceded their bet over a year ago, as per the edits in the lesswrong post.

The elders were right: Predicting really is easier if it's not about the future.

@mods This can resolve

@Primer which thing happened?

@ian Both conceded the bet

[Update 4 (12/23/2023): Tamay has now conceded.]

[Update 3 (3/16/2023): Matthew has now conceded.]

as per the lesswrong post.

I just wrote a retrospective on the bet. I think it's highly likely that we'll lose, unless something weird happens, like people stop publishing on all the benchmarks that we used for the bet. I think what happened is that (1) my AI timelines were simply too long, and (2) I wasn't thinking very carefully about the milestones, and didn't realize that extrapolating past trends reveals that we are expected to lose unless progress slows down by a lot.

Related questions

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules