Will the next major LLM by OpenAI use a new tokenizer?
43
Ṁ12232025
76%
chance
1D
1W
1M
ALL
The GPT-2 model used r50k_base: vocab size = 50k
The GPT-3 model used r50k_base: vocab size = 50k
The GPT-3.5 model used cl100k_base: vocab size = 100k
The GPT-4 model used cl100k_base: vocab size = 100k
Get Ṁ1,000 play money
Sort by:
@firstuserhere So YES if there's a GPT-4.5/5 that uses a tokeniser not on this list, and NO if there's a GPT-4.5/5 that uses a tokeniser that is on this list?
Related questions
Related questions
Will the most interesting AI in 2027 be a LLM?
41% chance
Will Google have a better LLM than OpenAI by 2025?
35% chance
Will OpenAI have the best LLM in 2024?
59% chance
Will there be a OpenAI LLM known as GPT-4.5? by 2033
35% chance
Will OpenAI release a tokenizer with vocab size > 150k by end of 2024?
42% chance
Will OpenAI release a tokenizer with more than 210000 tokens before 2026?
24% chance
Will OpenAI reveal a textless LLM before 2025?
20% chance
What is the next OpenAI LLM logo color?
What will be true of OpenAI's best LLM by EOY 2025?
Will openAI have the most accurate LLM across most benchmarks by EOY 2024?
39% chance