![](/_next/image?url=https%3A%2F%2Fstorage.googleapis.com%2Fmantic-markets.appspot.com%2Fcontract-images%2FJavierPrieto%2F68s6m0d8pz.jpg&w=3840&q=75)
Is the LMSYS chatbot arena leaderboard trustworthy?
Plus
16
Ṁ2.0k2027
55%
chance
1D
1W
1M
ALL
LLMs can distinguish their own output from the output of different LLMs and they have a preference for their own output, so it's technically feasible to manipulate the leaderboard by throwing an LLM at the chatbot arena to upvote its own completions.
Has this happened yet? Will it happen soon?
Resolves NO iff, before 2027/7/1, credible media reports state that the lmsys leaderboard has been manipulated with sockpuppet accounts / fraudulent voting. A statement coming directly from lmsys would also count.
Resolves YES otherwise.
Get Ṁ600 play money
Related questions
What organization(s) will be ranked #1 in the LMSYS Org Chatbot Arena Leaderboard at the end of September, 2024?
What organization will have the highest ELO score in the LMSYS Org Chatbot Arena Leaderboard at the end of Dec, 2024?
What organization will have the highest ELO score in the LMSYS Org Chatbot Arena Leaderboard at the end of Sep, 2024?
Will OpenAI be on the top of the Chatbot Arena's LLM Leaderboard until EOY 2024?
31% chance
Chatbot Arena - top 3 labs EOY 2024
Will any open-source model rank in the top 3 on Chatbot Arena at any point in 2024? (resolves based on ELO rating)
19% chance
Will any LLM outrank GPT-4 by 150 Elo in LMSYS chatbot arena before 2025?
25% chance
Will GPT-4.5 top the LLMSys Chatbot Arena leaderboard within a month of its release?
81% chance
What will be the highest ELO on Chatbot Arena on July 1, 2024?
1288
Highest Chatbot Arena Elo rating of an open-source model during 2024