13 Choosing the Right AI Tools - SCAI024

Choosing the right AI tools for your film. In this lesson, we'll go through finding the right AI tools to create your film, what tools work better than others for certain things, from camera control to the action taking place on the screen, to the lengths of gens. In this lesson, I'm going to go over six different image to video tools. In AI, the tools are always changing and evolving. So it's best to choose which tool you want to use and to define that by a per project basis. And so at this current moment, the six tools that I'm going to go through are LumaRay 2, Kling 1.6, Minimax, SkyReels, Pica 2.2, and Runway Gen 3. When I go through these tools and I'm trying to figure out what I'm going to use for my film, I always look at what is going to be the most problematic shots and how the tools will perform with those. Because if the tool performs well with a very problematic shot, it's probably going to perform well with your easier, less intense motion and action shots. But that's not always necessarily the case. So if you're working within a budget and you can only get one or two of these tools, I strongly suggest figuring out which one you're going to get the most out of and is the most cost effective. If you are part of the CPPs for any of these tools, definitely use that to your advantage. That's the Creative Partner Program. Some of those come with unlimited credits. Some of them come with early access to new updates on tools. So it's great to utilize all the things that you have at your disposal. But in this initial test, I'm going to take the same image and run it through all six. So I'm going to first start with Luma's Ray 2. This is easily one of my most favorite models that's out right now. So the shot that I'm bringing in is my main character walking and he has this pod that floats behind him. So I want the camera to be tracking as he's walking forward and the pod floats behind him. Now in Ray 2, you also have an end frame. So this is great for especially when you're doing like filmmaking specifically, if you need to go from one point to another and you're able to create those images in mid-journey to then bring them in here and do that. It is so, so helpful. So here in our settings, we can see we're on Ray 2 and we're on 1080 and we're going to do it at five seconds. So the prompt that I'm going to do is The man's boots are walking forward. And sometimes you'd wonder like, why aren't you just doing the man walks forward? I have found, and this is my own experience, feel free to try anything. That's the other thing about AI is you just kind of tweak the prompts as you go. But I have found that when you're a little bit more specific on what you want walking forward or moving, it will perform a little bit better. So the man's boots are walking forward as the white pod follows behind him floating off the ground. Let's do tracking shot of... And then, you know, another thing in Luma is their camera control. So if you type in camera, you'll see all of these amazing camera moves. So move left, move right, up, down, push in, pull out, pan left, pan right, orbit. Orbit's always a fun one. Orbit right, crane up, crane down, and static. But in this one, I want it to be a tracking shot. That's not in there, but it usually does a pretty good job of understanding what a tracking shot is. So let's go ahead and run that. All right, now that the shot is gen, let's see what happened. So not a whole lot of good there. I do like the quality, but the guy is walking the wrong way and towards the pod. So we're going to try it again and tweak the prompt a little bit. So I'm going to do tracking shot of the man walking forward towards camera as the white pod hovers and floats behind him. And we'll see what we get with that. And on a lot of these, it's really about just genning over and over and over. At least in my case, that's what I do. And I've been able to get really good results that way of just repetition. And if the image looks like how this one did our previous one, tweak the prompt. And once you get one, even if it's not the one you're going to use, but the prompt gets a little bit closer and more dialed in, then you know you're on the right path. And then you just kind of keep rerunning that one, messing with it, trying it and see what happens. Okay, now that the gen is done, let's take a look. He's walking the right way, but now we get an actual man. And that's what I was kind of mentioning about earlier with the boots is when you give it something specific, it knows what to do. So we'll try this one one more time in Luma and just see what we get. Normally in my normal workflow process, I will do this over and over and over and over again. So I'm going to take this one, even though we didn't get it, but tracking shot of the man's boots walking forward. I'm going to add towards camera as the white pod floats and follow behind him floating off the ground. But as you can see in my library of shots here, I will run one shot over and over and over and over doing different prompts and things of that nature, just to really dial it in. Like I did one of this woman looking out the window and I kept getting the rain inside. So it took a little bit before I got one where it was done correctly. So a lot of this is a trial and error process. And that's another thing to take into consideration of defining which tool you're going to use because you don't want... This one ended up coming out really good. The water on the window is there, but there's no rain inside. This is a really good method to figure out which one you might end up having to do gens of this amount and defining which tool fits in the budget and which ones come with unlimited options and things of that nature. All right, now that the gen's done, let's see how it did. Still not great. I'm not loving that. So perfect reason to now go over to our next tool. We're going to go to Kling and take the same image into Kling 1.6. I'm going to start with the same prompt of the tracking shot of a man's boots or walking forward towards camera as the white pod floats and follows behind him floating off the ground. In Kling, there's also a negative prompt, which can be really good, especially if you're generating something and it's almost doing it, but there's maybe one thing that's wrong. This is great to come in here and put it in the negative prompt so that doesn't happen. You also have the option of 5 and 10 seconds. Always do it in professional mode. That's the higher quality model. Usually, I don't mess with the creativity slider too much when it comes to the filmmaking side of things. If you're doing something more experimental, might mess around with that slider. But in this particular case, because we're really just trying to maintain what our image looks like, but give it motion, I stick to just leaving it at the standard 0.5. A little bit better, we got the boots walking forward, but the camera's not moving, and the pod is just kind of staying still. I actually like everything that happened there, so I'm just going to generate it again with the same prompt, and then I'm going to make one adjustment, tracking shot of the man's boots. Let's go man's boots walking forward. The white pod follows behind him floating off the ground, and then I'll do this also one more time, but add tracking motion shot. These will be our three that we'll try here in Kling. Now that all of them have genned, still getting the same issue of the camera not moving, so we got kind of the same result in all three. Camera doesn't move, next thing I would try is tracking shot as the camera moves with the man's boots as they walk forward, and we'll try that one. So again, this is the benefit of taking one of your shots you know is more difficult and running it through, and a lot of this again is just rolling and rolling and rolling again. For me, I would probably roll in Kling and Luma Ray 2 anywhere from 20 to 50 to 60. I've done some as many as 65-70 gens per one shot, especially when it's my more difficult shots, because I know it'll eventually get there, and it's just tweaking the prompt, but it's again something you need to factor in with your budget and what tools you have available. Now if you have all tools available, what I always suggest doing is taking the image, taking the same prompt, run it through all six, make the prompt adjustments, see which one gets the closest, and just keep going through. No matter what, you'll eventually get to the shot you're looking for. See, there we go. Pod's not moving, but we're at least now getting closer that the camera is moving backwards as the man walks forward. So this again would take just some additional prompting to get the pod floating behind him, but it definitely got closer to what we needed it to do. Next is Minimax. What I love about Minimax is for $100 a month, you can have unlimited gens, and I mean you can just gen until you're blue in the face. So I'm going to take that prompt, since it worked really well, from Kling, put it into Minimax, and I'm going to gen this five times. If you see over here on their counter, you can do this up to five times. And if this initial gen doesn't work, Minimax also has great camera movement controls to where you can do all different kinds of things from left walking, upward tilt, right circling. This is in the new director mode. It's incredibly helpful. Then they've got the free selection to where you can just plug these in. So here's tracking shot. So if it doesn't work in, but normally with just the text prompt, it should register that. But yeah, we shall see how it does. All right, our Minimax images are animated. Let's see. Oh, right out the gate. This one's, I mean, does it perfectly. Let's look at that bigger. Yeah, that's really, really good. There's a little bit of artifacting in the boots, but the motion is really incredible. And it's got the pod following it perfectly. This one, not so much, but that first one is really impressive. I'm not a huge fan of that one either. So the first one was the one. I mean, that is really, really good. You got the motion, the jacket, and the pod. The pod following is really, really impressive how well it did that. Like I said, there is some artifacting issues, especially over here. But overall, the motion is incredible. And that's kind of, again, when you're comparing these tools. For instance, I find Kling and Luma usually have less artifacting, but Minimax usually has a little bit more in this current model, but it listens to motion really well. And some of that artifacting can be cleaned up when we take it into post into Topaz Labs. So next, we're going to go to SkyReels. SkyReels is a fairly new model, but I've been really impressed with everything that it's been doing. All right, I'm taking that same prompt that worked really well in Minimax, and we're going to put it into SkyReels and see how it does. In this, I'm going to generate it three times just so we can see a few different options. All right, now that all three of our images are done, let's see how they did. That looks really good. Almost little to no artifacting. I mean, that's incredible. The pod's subtly floating. I mean, that one's great. I would use that for sure. This is the next one. Pod really isn't moving. Feet walking are great. Love all the detail. That looks really clean. Would like the pod to move a little bit more in that one. And there's my favorite, I think. You got great foot motion walking forward, little to no artifacting, and the pod is following behind perfectly. Yeah, that's a great shot. So right now, SkyReels is in the lead for this particular shot. That was very impressive. So let's switch over to Pico. All right, we've got the image in Pico. Going to use that same prompt. That prompt seems to be doing well. We'll do the same test with Pico. We'll hit it with three gens and just see how they come out. All right, the Pico gens are done. Let's take a look at how these did. Not too bad. There's a weird glow that I wouldn't necessarily want. That's definitely not what I wanted. I mean, it looks interesting. Also not what I wanted, guys walking the wrong way. But I do like this kind of little camera shaking at it. So this was interesting. This was cool for sure. But overall, still going to prefer some of the previous iterations that we've had. So lastly, we're going to go to Runway Gen 3. And we're going to use the same prompt where we had success in both Minimax and Sky Reels. So let's see how this prompt does in Gen 3. And we'll give it the same test of three gens. All right, now that the Runway gens are done, let's see how they did. The guys walking the wrong way. Weird light elements here. And I don't know what's happening with the jacket. That's very, very strange. This would be a non-usable. Same again. The whole thing's kind of breaking apart and the lighting's changing quite a bit. For me personally, in this initial test, I would have ended up using the Sky Reels image. I think this one, or actually I believe it was this one. Yeah, this one right here. Was probably my favorite of all of them. When I'm working on a project, I would either go with Luma, Kling, Minimax, or Sky Reels. I love Pika and Runway. I think they're great in other aspects. But for the cinematic image to video, I think these four kind of stand above the rest. But this again comes down to what shot works best for you. And what elements are you looking for? What details? All of those things are important. From the artifacting, the camera motion, what's happening in the shot. And then on top of that, budget. What fits in the budget for the project? This would conclude lesson five. I really hope this helps give you guys some insight into the tools. Seeing these different examples and what works well and what doesn't. How messing with prompts along the way, especially as you get a little bit closer, fine-tuning those and understanding. A lot of this is just having to gen it a lot before you get the shot you're looking for. Sometimes you might get it right away. And sometimes you might have to gen it over and over and over and over before you get the look you're going for. And that's the end of this lesson. I will see you on the next one.