On January 1, 2027, a Transformer-like model will continue to hold the state-of-the-art position in most benchmark | Manifold

On January 1, 2027, a Transformer-like model will continue to hold the state-of-the-art position in most benchmark

Plus

38

Ṁ4122

2026

82%

chance

1D

1W

1M

ALL

Tracking external bet: https://www.isattentionallyouneed.com/

This question is managed and resolved by Manifold.

Get

1,000

and

3.00

Sort by:

What constitutes Transformer-like?

Is it all, or some, of these?

Token inputs, token outputs
Positional encoding
(MH QKV SDPA -> MLP) layers
Residual connections

bought Ṁ50 NO

https://arxiv.org/abs/2501.00663

Titans: Learning to Memorize at Test Time

Abstract page for arXiv paper 2501.00663: Titans: Learning to Memorize at Test Time

@JaundicedBaboon Let's see if it can leave the academia limbo where the Mamba's got stuck...

https://x.com/_philschmid/status/1846452029582959012

@MalachiteEagle Transformers, yes. Attention, not really:

followed by two shared attention blocks.

https://blog.rwkv.com/p/eagle-7b-soaring-past-transformers

It even links the wager page in its bragging points:

All while being an “Attention-Free Transformer”

I thought this wager was made today haha

Does this strictly resolve to the outcome of the bet?

@FranekZak Yep. Whatever details they come up with are in.

Related questions

Will transformers still be the dominant DL architecture in 2026?

+18% 1d79% chance

Will Optimus be used on the Optimus assembly line before the end of 2025?

-7% 1d28% chance

Will there be over 10,000 Optimus robots working at Tesla before 2027?

Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.

Will a Chinese AI developer announce a model rivaling o3 performance by February 2025?

Will Transformer based architectures still be SOTA for language modelling by 2026?

+8% 1d78% chance

A major ML paper demonstrates symbolic-enhanced transformer successor outperforming standard transformers by March 2025

Will the most capable, public multimodal model at the end of 2027 in my judgement use a transformer-like architecture?

Will superposition in transformers be mostly solved by 2026?

Will an AI achieve >85% performance on the FrontierMath benchmark before 2027?

Related questions

Will transformers still be the dominant DL architecture in 2026?

Will Transformer based architectures still be SOTA for language modelling by 2026?

Will Optimus be used on the Optimus assembly line before the end of 2025?

A major ML paper demonstrates symbolic-enhanced transformer successor outperforming standard transformers by March 2025

Will there be over 10,000 Optimus robots working at Tesla before 2027?

Will the most capable, public multimodal model at the end of 2027 in my judgement use a transformer-like architecture?

Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.

Will superposition in transformers be mostly solved by 2026?

Will a Chinese AI developer announce a model rivaling o3 performance by February 2025?

Will an AI achieve >85% performance on the FrontierMath benchmark before 2027?

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules