Feel free to propose more precise criteria for "something like activation addition".

• Activation Addition is the canonical example I have in mind here

• The method should intervene on model activations, not model weights, so I think I'd exclude things like Task Arithmetic. If you think it would be more interesting to include them please comment, I might change my mind.

I will resolve the question by the end of the year 2024 - if later than 2024 it turns out that models were using such methods, but we didn't learn that in 2024, then the question will resolve as NO.

I'd count an official announcement by the lab as sufficient to resolve as YES. I'd also count sufficiently credible rumors as sufficient.

For example:

• I'd count the SemiAnalysis report on GPT-4 as sufficiently credible evidence that GPT-4 used MoEs

• If this question was about "something like Speculative Decoding", I'd also resolve this question as YES because my impression is that the most informed people think that big labs like OAI and Anthropic use something like SD, even though AFAIK they haven't confirmed this officially (?) [ok after checking this was also reported in the SemiAnalysis report on GPT-4]

If labs experiment with these methods but don't find them promising enough to use them on their best models the question resolves as a NO.