Indeed, but if you look at the horror show from RunwayML Gen2:

https://satinminions.com/RunwayML-Violet-Clean.html

This has very impressive consistency in comparison.

Right now I think we're getting the worst end of AI, where it's convincing enough on the surface to blow up social media, but not really practical enough for a big improvement in production.

There is a (bright?) future where a single person will be able to make a complete film using these tools. I don't know what it'll be like when there's a thousand or ten thousand new shows released every year but it sounds like we will live in interesting times.

My worry is that the best tools will remain proprietary and lawfare will be used to remove the open source alternatives.

Some people think that the current tech will cap out and never really be useful - I think this is cope. The capabilities are clearly already there, it's just a matter of controlling them.

It is unnerving for sure - especially when I go to deviantArt or rule34 or even google images and the entire page is destroyed with ai garbo. I've said this before but that's why I feel the need to mess with it, because I need to understand the threat and how to use it.

Oh I guess my other big worry is that we'll enter a completely post-fact world and it'll be impossible to find a verifiably real image of anything. Every book will be rewritten and every image repainted, all recently recorded history will be in doubt, etc.

AI video just off doing its own thing.

You can nitpick the hands and feet but if you saw this a few years ago you'd probably think it was some random bad TV show.

The future is gonna be weird.

This is a result from Luma Labs AI image-to-video. It is implemented as web page where you can provide a single image and text prompt.

Here I used this image:
https://satinminions.com/Alison-Jessica-Rabbit-Side.html

The stability and movement are impressive and it has a good understanding of materials such as the sequins and satin gloves. It tends to favor a photo realistic style, 3D rendered style, or a generic flat anime look depending on how you prompt it. Sometimes it will cut to a completely different scene, like this:
https://satinminions.com/LumaLabs-Combat-Mode-Cut.html

We're fast approaching the point where generative video AI can replace stock footage, things like "man using computer" or "drone shot of new york city at night" can be completely convincing. And of course memes where consistency doesn't matter are great. But I don't think the current model architectures will be able to replicate a style or character without specific training.

The current target audience seems to be the zero-skill user, where you only have to provide the bare minimum - because that's what gets the most engagement. As a "professional" though, I would much rather see tools that required more advanced inputs - for instance, input a complete line animation and a single colored frame and it would propagate that coloring through the animation.

Another result from Krea AI, using some of the same keyframes I fed to ToonCrafter here:
https://satinminions.com/Lighter-Chains-4-ToonCrafter.html

It has impressive stability in the face and hair, but it can't do the kind of scene interpretation that ToonCrafter can.

This is a result from Krea AI image-to-video. It is implemented as web application where you can upload keyframes and prompts on a timeline. I used these two images as keyframes:

https://satinminions.com/Alison-Jessica-Rabbit.html
https://satinminions.com/Alison-Jessica-Rabbit-Side.html

It's eye catching and very smooth (at least in the face), but it does not follow the original keyframes or style very strongly. I think this could be tweaked to get better results, but my "3 minutes" of free trial video generation got used up in 20 seconds, and that doesn't exactly encourage me to purchase more time.

Having a timeline with keyframes is a step in the right direction, but the hard push towards content control and monetization seems premature for something that's not really cut out for professional use.

This is a result from RunwayML Gen2 AI image-to-video. It is implemented as a web service, so it costs money, it can be slow, there is content filtering, and you're at their mercy for data privacy and access.

It generally struggles to maintain a consistent form, with characters often melting into someone else if there is significant animation. It also can't replicate a style, which is why I used the stable diffusion edit of this character as the image prompt, since it is a more generic photorealistic style.

They have announced Gen3, but it is not available to the public yet. From the results they've shown, I'd say it has potential to replace stock video for corporate ads and video essay filler, but it's not quite there yet for animation production.

Pika Labs AI image-to-video does a pretty decent job at realistic smoke, water, rain, fire, that sort of thing.
The intent seems to be to create cinemagraphs.

This AI image-to-video from Pika Labs. You provide a single image and a text prompt and roll the dice. It's implemented as an online service via a discord bot so you sometimes have to wait a long time and there is NSFW filtering.

This was one of the only results I managed to get with a coherent movement that didn't morph into something strange. Most of the time you get the windy hair/fabric effect applied to random parts of the image or no animation at all.

Maybe if it could run locally, you could run it hundreds of times and cherry pick good results, but as it is now, it doesn't seem to have any use at all.

You may be interested in this video:
https://satinminions.com/Satin-Minions-The-Offering.html

Also I realized I named this "Lighter-Chains-4-ToonCrafter" when it is based on LC5...

This is another result from ToonCrafter, an AI frame interpolator, this time fed with panels from the opening scene of Lighter Chains Volume 5. There's some weird bits of course, but there's a lot of potential here for automating the "boring" parts of an animation where you just need some blocking and idle animations.

As a storyboarding aid, this can also show what amount of limited animation you can get away with - some of the shots work surprisingly well.

I've had some interesting results upscaling the frames from this in stable diffusion, but I want to see if doing some quick fixes to the animations or training a lora on the source panels can help things.

Well prepare to sit wrong because I have tried several image-to-video AIs I'll be posting the results and my thoughts this week.

The goal isn't to make the best finished work right off the bat, but to test the technology and see what it can do - good and bad. Part of learning a new tool.

Or sometimes it's just a novelty.

I made this using a new open source AI tool called ToonCrafter that can interpolate an animation between two given key frames.

I used various crops and reverses of these two images as keyframes:
https://satinminions.com/Alison-Jessica-Rabbit.html
https://satinminions.com/Alison-Jessica-Rabbit-Side.html
The results are a little strange at times, but given that these are not really animation keyframes, I think there is some potential here.