It can be done somewhat now. Just depends on how high your standards are for time consistency. Thoughts @Sss19971997 ?

https://arxiv.org/pdf/2311.18834.pdf

Google's "deep dream" could run in real-time several years ago. It's 100% AI generated content, but I'm guessing you don't count it - why not?

@Retr0id I thought that was an image generator. I am talking about a video generator that can generate almost infinite long video while preserving some time consistency

@Sss19971997 People made videos with it, e.g. https://www.youtube.com/watch?v=Jik4_Wt3uiM I guess it's like the video equivalent of markov chains - not useful for much, but imho hard to say why it's not AI

@Retr0id Yeah, It is super hard to give a super clear criterion

@Retr0id But the video you showed me is not what I am talking about. Shall I add quality>SD1.5 as part of the criterion or something?

I know it's not what you mean, but there's nothing in the current resolution criteria that say it doesn't count :)

I guess it's always going to be subjective, but maybe sora would be a better point of comparison? (people have made deepdream-like videos based on stable diffusion also, and you could plausibly do realtime with enough GPU)

this is subjective

@ZoravurSingh When predicting future, ambiguity is inevitable. Would you please give some advice on how to make the resolution rules clearer?

@Sss19971997 perhaps a criteria on how important quality/resolution is? There are probably already models that could do this for 144x144

@ZoravurSingh time consistency across the stream is the key criterion

I don’t think resolution matters that much. Let’s say 256x256x3x24 frames per second

This could be cool as a concept though. If you could feed choices in from the audience.
You could have some prior setup prompts about the environment and then have audience input the choices about what happens in the story/stream.

Have entity x do a,b or c. Have audience vote on it and have it generate in real time resolution with time consistency.

https://github.com/cumulo-autumn/StreamDiffusion

Don't think that is configurable in a way that meets the criteria, as I don't see it actually generating video.
As in there is no interdependence between what was generated in time before to frames that come after. So it would appear to classify itself more as a prompt filter.

Unless that's somehow what OP meant.

Probably technically possible to setup with a very low quality of output: similar to the early models output, nothing close to Sora. And the description doesn't say anything about the quality.

Yea you could get a progressive generation setup now with some compute in real time now, it would look like dog turd and not be time stable. But it would qualify I suppose.

Quality wasn't a requirement.

He does list time consistency which old models don't REALLY have (IMO). And Sora resolves all the frames at once, so isn't real time applicable.

Hence my low probability.

https://www.youtube.com/watch?v=zD9wofGof80

resolves yes

That's not video generation... ?

@Unown No… I am talking about generated with stat model, anything involve PDE-based physics engine does not count

<10% for sure. Unless I'm missing something.

Aww I mean you could probably whip up some real time video generation now. It would just look like turd.