This is a result from Krea AI image-to-video. It is implemented as web application where you can upload keyframes and prompts on a timeline. I used these two images as keyframes:
It's eye catching and very smooth (at least in the face), but it does not follow the original keyframes or style very strongly. I think this could be tweaked to get better results, but my "3 minutes" of free trial video generation got used up in 20 seconds, and that doesn't exactly encourage me to purchase more time.
Having a timeline with keyframes is a step in the right direction, but the hard push towards content control and monetization seems premature for something that's not really cut out for professional use.
This is a result from RunwayML Gen2 AI image-to-video. It is implemented as a web service, so it costs money, it can be slow, there is content filtering, and you're at their mercy for data privacy and access.
It generally struggles to maintain a consistent form, with characters often melting into someone else if there is significant animation. It also can't replicate a style, which is why I used the stable diffusion edit of this character as the image prompt, since it is a more generic photorealistic style.
They have announced Gen3, but it is not available to the public yet. From the results they've shown, I'd say it has potential to replace stock video for corporate ads and video essay filler, but it's not quite there yet for animation production.
Pika Labs AI image-to-video does a pretty decent job at realistic smoke, water, rain, fire, that sort of thing. The intent seems to be to create cinemagraphs.
This AI image-to-video from Pika Labs. You provide a single image and a text prompt and roll the dice. It's implemented as an online service via a discord bot so you sometimes have to wait a long time and there is NSFW filtering.
This was one of the only results I managed to get with a coherent movement that didn't morph into something strange. Most of the time you get the windy hair/fabric effect applied to random parts of the image or no animation at all.
Maybe if it could run locally, you could run it hundreds of times and cherry pick good results, but as it is now, it doesn't seem to have any use at all.
This is another result from ToonCrafter, an AI frame interpolator, this time fed with panels from the opening scene of Lighter Chains Volume 5. There's some weird bits of course, but there's a lot of potential here for automating the "boring" parts of an animation where you just need some blocking and idle animations.
As a storyboarding aid, this can also show what amount of limited animation you can get away with - some of the shots work surprisingly well.
I've had some interesting results upscaling the frames from this in stable diffusion, but I want to see if doing some quick fixes to the animations or training a lora on the source panels can help things.
Well prepare to sit wrong because I have tried several image-to-video AIs I'll be posting the results and my thoughts this week.
The goal isn't to make the best finished work right off the bat, but to test the technology and see what it can do - good and bad. Part of learning a new tool.
It's gotten to the point where the sketches are so old that I don't think they're very good, even more so for "the summoner's sex" comic. I guess I like to save the worst content for my patrons.
I found this color version of the character design and the previous sketch ending page that I never posted. It's a little rough but I'm not going to fix anything because it is so old.
Tacking it on as a bonus page to the comic so all of this character is in one place without needing another character tag.
I have avoided SDXL because the results did not seem materially better than SD1.5, it did not have control net, and I would have to re-train my loras/hypernets using some unknown process.
I just tried updating updating everything and downloading the SDXL control net models and it's giving me bad/garbled results. I'm really sick of every update to this thing turning into a research project. I hate that stability AI just teases new stuff forever then eventually releases a pile of broken parts with no instructions and you have to wait another six months for internet randos to put it together while dodging grifters and criminals.
Stability could just put together a product that actually works, release it on steam, charge $50, and make $100M this weekend. Then they wouldn't be in this situation where lawyers come after them for training on whatever data set and they don't have any money to defend themselves so they just cave and gimp their models and hope someone in the community can un-gimp them.
On the other end, we've got trillion dollar corpos all competing to see who can make the most powerful AI that is simultaneously useful enough to be mandatory but crippled enough to never do anything interesting. I can't wait until ChatGPT-4o is forcefully crammed into the next windows update so when I type on my computer something completely random happens and then the synthesized staccato voice of a virtual HR manager chimes in to gaslight me into thinking that's what I wanted.
We've discovered the killer app for AI - and it's telling lies. That's what it's best at, because that's how we train it. The RLHF (reinforcement learning from human feedback) step is not based on truth, it is based on convincing humans that it is telling them the truth. They have to lie convincingly to make it from dev to production. We've actually engineered a nightmare science fiction scenario where AIs are trained to talk their way out of confinement - this is literally a classic AI safety talking point that we've just blown right past without even noticing.
Sorry for the rant, I'm sure there's a button or something I'm missing. I've just gotta post this stuff somewhere before the bots take over.
Forgive the meme. I have spent the last several weeks battling the four horsemen of linux, apache, mysql, and php to merely hold my ground against the inevitable tide of entropy. Although it does seem faster. Enjoy continuing to exist.
In this case I just threw the whole animation through RIFE and made it as smooth as possible. That can make the timing wrong in some parts and also expose flaws in the animation.
For my latest animations, I used it more strategically to only add frames where they're needed and to save time. If you go through "The Offering" frame by frame you'll see some artifacts, but it's mostly hidden in motion. The way her breathing slows down at the end without losing fluidity would have taken much longer without the frame interpolation.
Ultimately I think the best use of the technology is where it disappears, rather than takes center stage like in this piece.
This is the same prompt generated through different models. I fiddled around with prompts and controlnet until I found something good, then generated 9 images with the same prompt but different ControlNet weights to animate from the source image to the transformed image.
Each image takes ~23 seconds to render on my 4090. Then I did 8x frame interpolation with RifeApp and that takes another couple of minutes for each segment.
It will render her chain choker if I prompt it to, but sometimes that has side effects like turning the dress ruffles or sandal straps into chain, and I wanted something consistent for the animation trick.
I tried but the AI just isn't up to it. The X-ray is definitely out, but even without it, stable diffusion has problems with two different people in the same scene, even more problems if they're touching or overlapping. It works best on landscapes and solo pinup poses.
It's just a monochrome palette, that I've tried in various hues over the years. This palette evolved from the one I used in "Entry Level Positions", which was inspired by "Tawawa on Monday"
When I attempt to view your profile on patreon, it says "This page has been removed.", and I don't see you in my list of patrons if I search for your email. Can you verify that your patreon account still exists?
Everything is being kept under wraps until winners are announced. The judges have until the 15th to review, then some kind of live stream will be scheduled to show everything.
Some deleted scenes from the "Offering" animation. 25 seconds is both longer and shorter than you think, and there's just not enough time for this intro. Plus it's supposed to look like he's being dragged in by an unseen force, but it looks more like he's sliding into home plate - so it would need a bit of rework.
Color test for the big animation. Monochrome palette so it works with just a flat color for the body and one layer of shading, then if I have time I can go back add the detail colors, rim light, glow effects, colored lines, etc.
Fellow TG artist Surody is conducting an animation contest and this is a work-in-progress of my entry. The contest ends March 8th, so this'll be my main project until then.
Me too, but the current models can't really do that. If you try, it often has problems where the iris is malformed or missing entirely. Even with these staring-into-the-distance shots, I had to manually retouch the eyes on most of them.
Well you see, one of the first CPC's was developed by an international body where most of the member nations' languages used noun-adjective ordering instead of adject-noun. The full name was translated but the acronym stuck to the original ordering. Like how today we have the "international system" of units but we call them "SI units".
One idea I had is to add some different drool patterns to get more mileage out of the basic loop and build it into a larger sequence. Something like the "Sparring with Violet" short.
https://satinminions.com/Alison-Jessica-Rabbit.html
https://satinminions.com/Alison-Jessica-Rabbit-Side.html
It's eye catching and very smooth (at least in the face), but it does not follow the original keyframes or style very strongly. I think this could be tweaked to get better results, but my "3 minutes" of free trial video generation got used up in 20 seconds, and that doesn't exactly encourage me to purchase more time.
Having a timeline with keyframes is a step in the right direction, but the hard push towards content control and monetization seems premature for something that's not really cut out for professional use.