Same as this market but closes and resolves end of 2024
https://manifold.markets/journcy/will-this-yudkowsky-tweet-hold-up
"This market resolves YES if at close (end of **2024**) my subjective perception is that this was a good take (https://twitter.com/ESYudkowsky/status/1563282607315382273) --e.g., AI-generated video really is that good--and NO if it seems like Eliezer was importantly wrong about something, e.g., AI-generated video still sucks, or still couldn't be the cause for serious doubt about whether some random moth footage was made with a camera or not."
1,000
3.00This reddit post is evidence in favor of a YES resolution IMO https://www.reddit.com/r/ChatGPT/s/lSCArBlw5d
@GeorgeVii can you share your view on the market as of now? Does the video released today from Meta Movie Gen meet your criteria to resolve yes?
I would like to buy more No but I have a lot of uncertainty about your view on the state of the art.
@aaron how can a single (likely cherry picked and tweaked) demo resolve a claim about videos in general?
@AIBear This is implicitly a question about cherry picked videos, no? It's about "if a realistic looking video goes viral, was it likely made by an AI?". Someone who wants to make a viral video can generate a lot, for many prompts, tweak them a bit, etc before releasing, and I think it still meets the spirit of Yudkowsky's prediction
@NeelNanda I think you're using cherry picked in a different sense: the demo was all the videos Meta found most impressive, which would be different from generating many potentially viral videos for a given prompt and picking the best one
@dflz No, I'm using it in the same sense. If someone wants to make a viral video, they can try many prompts, and generate many videos for each prompt, and pick the best out of all of them. Or just post several, and eventually one will go viral.
You also get the selection effect that, even if no one viral video creator is doing this, if there's thousands of them, you'll only see the videos that go viral, which is an extreme form of cherry picking for impressive videos.
@NeelNanda I mean the demo itself could be cherry picked so that only videos that look good for the model
For instance, if a story telling robot could only write stories about cats, and the demo only showed cat stories, that would be cherry picking.
Compare to a model that could do a great story 1/3 of the time, and then cherry picking the great third of stories.
In this case, I think it's about cherry picking capabilities of the model, vs cherry picking an individual attempt
@dflz Also to add, if creating one good video requires $100k in cloud credits and human evaluating 800hours of videos, you likely don't need to worry that a typical viral video was produced this way. But I can totally imagine Meta spending an order of magnitude more to produce a demo.
@NeelNanda Another part of my concern is procedural. The model is not out, you currently don't have to worry abou AI generated viral videos, therefore the question shouldn't resolve at this time. Discussing whether the question should resolve YES if the model is released and performs this well (or whether at least somewhat public release is required) makes sense. Resolving now would however be premature.
There's a long history of AI companies faking their demos, so some caution is warranted.
@aaron I would be really surprised if the meta model resolved this positive, since it is about as good as Kling 1.5 which has been publicly accessible for ~1 month and has not caused a positive resolution.
@aaron I think there should be an AI-generated "insect flying video" or at least a "bird flying video" that is as good as the original video, and I haven't seen a model that does that. Having a model that can create beautiful mountains or fish is impressive, but it's not what the question is about.
@AIBear The cost point is fair, and I'm not arguing that this should be resolved YES now, I agree that's silly before the Meta model is publicly available and can be tested.
I am just arguing that the bar for "many viral videos are AI generated" is way lower than "this thing will reliably give you a fantastic video on a reasonable prompt in a handful of attempts", because the process of virality and commercial incentives give you a ton of selection power beyond an individual, not very motivated, user of a model
@NeelNanda none of the models publicly accessible could produce that moth video (without using some additional trick like first rendering it 3d and then using vid2vid). I strongly suspect Sora/Meta would do no better.
@AIBear There is no requirement that all videos 'in general' must be of this quality. Only whether AI can generate a video that is indistinguishable from a natural video by your average Twitter audience.
@RiskComplex @AIBear @dflz @aaron @NeelNanda @Pazzaz I questioned how this would be resolves months ago:
Is this market about whether this was a good take regarding 2024 (1) or 2026 (2) or whether Yud thinks he was "right" regarding 2024 (3) or 2026 (4) or whether @GeorgeVii thinks this was a good take regarding 2024 (5) or 2026 (6) or whether @GeorgeVii thinks Yud was "right" regarding 2024 (7) or 2026 (8) or whether @GeorgeVii thinks Yud thinks this was a good take regarding 2024 (9) or 2026 (10) or whether @GeorgeVii thinks Yud thinks he was "right" regarding 2024 (11) or 2026 (12) or whether the tweet is generally regarded as "right" in 2024 (13) or 2026 (14) or whether Yud or @GeorgeVii think the tweet is generally regarded as correct in 2024/2026 (15-18)?
Maybe noteworthy: There was no response, so we might need to take into account some mod resolving this according to how they interpret the question.
@dflz I think some of it must be the uncertainty introduced by the question being resolved on the subjective views of one person.
To me, AI video generation is still incredibly far removed from producing videos of the quality seen above, there doesn't seem to be that much progress in the space - but who knows how it will be resolved.
@LukasTilmann Yeah, from my perspective as a (very minor) yes holder, I'd already say that the EY tweet was a good take. It's definitely true that if you stare closely at videos generated by AI you can notice things that are off, but I'm not sure that precludes the take being "good" this year.
So it does seem like a lot of what this question has left is how will GeorgeVii interpret the wording of this market. Like, I don't think AI video generation "still sucks" (as posted by nic below: https://x.com/GoogleDeepMind/status/1808888487975776520). But I agree that it isn't yet perfect.
Though I do wonder how people would do at categorizing videos like https://x.com/GoogleDeepMind/status/1808888487975776520/video/1 as AI
generated or not. Perhaps people who are really in-the-know with respect to signs about AI generation would be able to quickly tell, but I'm guessing if I showed that to my mom alongside some other real underwater footage, she wouldn't be sure which one was fake. Was EY's take meant to mean "can fool sleuths who are good at figuring it out", or should it be interpreted more liberally.
Honestly I don't envy GeorgeVii on trying to resolve this.
Tiny nitpick, but I believe the resolution depends on the creator of this market (https://manifold.markets/journcy/will-this-yudkowsky-tweet-hold-up), but your point still stands.
Some more stuff coming out of DeepMind:
https://x.com/GoogleDeepMind/status/1808888487975776520