Did OpenAI use MUP for zero shot hyper-parameter transfer in GPT-4?
Basic
5
Ṁ1692025
81%
chance
1D
1W
1M
ALL
Maximal Update Parameterization is technique published last year by Yang et al. at Microsoft. https://arxiv.org/abs/2203.03466
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Sort by:
@firstuserhere interesting that it is in the bibliography, although the reference in the first image is from a different section of the report with its own bibliography (that [16] actually refers to "DALL·E 2 Preview - Risks and Limitations.").
So the muP paper is in the bibliography, but not referenced anywhere.
@Stefan yep, and even then it's not actually used in gpt-4, the report only mentions the red team to have used the paper?
Related questions
Related questions
Will OpenAI open source the weights to one of the GPT family models in 2024?
3% chance
Will OpenAI release GTP-4.5 before GPT-5?
58% chance
Will OpenAI change their naming scheme (GPT-X) with the successor to GPT-4? (Ṁ200 subsidy!)
14% chance
Has openAI intentionally made chatGPT lazy to save inference costs?
21% chance
Will the next LLM released by OpenAI be worse than GPT-4 at MMLU?
16% chance
Did OpenAI intentionally handicap GPT4's image modality's ability to identify people?
83% chance
Will OpenAI's autonomous agent be based on GPT-4?
19% chance
Will OpenAI suggest GPT-4 is AGI?
4% chance
Will OpenAI provide access to GPT-4 weights to academic researchers not affiliated with OpenAI, by 2025?
14% chance
Will OpenAI's GPT-4 API support image inputs in 2024?
97% chance