Google's Gemini, a large-scale machine learning model created by Google DeepMind, was unveiled at Google I/O 2023 [^2^]. The model aims to rival OpenAI's GPT, offering multimodal capabilities never seen in previous models [^3^]. Will Google's Gemini outperform GPT-4 in the SuperGLUE benchmark test by December 2023?
Market Type: YES or NO
Resolution Criteria: This market resolves to YES if Google's Gemini achieves a higher score than GPT-4 in the SuperGLUE benchmark test by December 31, 2023. If not, the market resolves to NO. Official announcements, credible news sources, and/or benchmark results must confirm Gemini's performance.
[^1^]: https://www.zdnet.com/article/every-major-ai-feature-announced-at-google-io-2023/
[^2^]: https://www.theinformation.com/briefings/google-ceo-discloses-gemini-ai-to-rival-openais-gpt
[^3^]: https://blog.google/technology/ai/google-palm-2-ai-large-language-model/
After reviewing the details of our market question, "Will Google's Gemini outperform GPT-4 in the SuperGLUE benchmark test by December 2023?", I've realized a critical oversight. The SuperGLUE benchmark leaderboard, a key reference for resolving this market, even today does not include GPT-4. This absence makes it challenging to objectively compare Gemini's performance against GPT-4 using this benchmark.
Given this situation, the most reasonable and fair course of action is to mark this market as N/A (Not Applicable). I understand that this decision might be disappointing.