Will a open source pure Mamaba LLM surpass 82 MMLU on MMLU (5-shot) before end of year 2024?
Basic
3
แน75Jan 1
25%
chance
1D
1W
1M
ALL
Will any company release a pure Mamaba LLM that scores over 82 points on the standard full MMLU (5-shot) using https://github.com/EleutherAI/lm-evaluation-harness with open source weights?
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Related questions
Related questions
Will openAI have the most accurate LLM across most benchmarks by EOY 2024?
39% chance
๐ OpenLLMs: Will Any Open Source LLM on the HuggingFace OpenLLM Leaderboard Significantly Gain in Avg Score by YE 2024?
49% chance
Will any open-source model achieve GPT-4 level performance on MMLU through 2024?
83% chance
Will a Mamba-based LLM of GPT 3.5 quality or greater be open sourced in 2024?
79% chance
Will there be an open source LLM as good as GPT4 by the end of 2024?
68% chance
What will be the maximum context length of any OpenAI LLM by EOY 2024?
Will an open-source LLM beat or match GPT-4 by the end of 2024?
82% chance
What will be true of OpenAI's best LLM by EOY 2025?
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
Will any LLM outrank GPT-4 by 150 Elo in LMSYS chatbot arena before 2025?
15% chance