How can AI help you expand a scene in the edit? In this video, I share how I’m using Higgsfield in post-production on my new film—from creating extra coverage to building VFX shots.
I walk through how AI image generation and AI video tools help me create additional shots, giving me more options to shape a scene, adjust its pacing, and expand what’s already been filmed. I also show how I’m using these tools to create visual effects for the film.
If you’re exploring AI filmmaking, looking for a Free AI video generator, or comparing services to find the Cheapest AI Platform for your workflow, this is a practical look at how I use Higgsfield on a real film project. The main tool I used is the Supercomputer which is an AI super-agent which uses all the tools that Higgsfield has to offer in order to achieve what you ask it to.
Where would AI tools fit into your filmmaking process? Let me know in the comments.
Chapters: 00:00 How is AI used in VFX Industry? 01:03 My Shark Film Problems 03:35 Location and Character Sheets 03:55 Why is Higgsfield the All-in-One Platform? 04:48 What is the Supercomputer? 07:25 How to treat the prompt writing process? 09:15 Why should you ask the AI Agent to ask you questions? 12:20 How can I save money on AI credits? 14:09 Why is your AI video so bad? 15:07 How to enhance the prompt 16:22 How to avoid AI Slop? 16:58 Where AI can't compete with Humans? 18:00 Why do some AI video prompts fail?
---------------------------------------------------------------------------------- Remember, in filmmaking there's no problems, just a lack of solutions!
My name is Tom Antos. I am a film director and cinematographer with over 20 years experience in VFX & animation.
-------------------------------------------------------------------------------------------------------- DISCLAIMERS: The opinions expressed in this video were not influenced by or paid for by any outside individual or company. Higgsfield did not ask me to say anything in particular for this video nor did they review the video prior to it being posted online.
AI Generated·May contain errors
Video Summary
AI Generated
Veteran VFX supervisor Tom Anthos demonstrates a professional workflow for using AI tools to fix production errors and generate missing coverage. By utilizing real-world references and an iterative prompting process, he integrates AI-generated visuals into a short film while maintaining cinematic integrity and avoiding the generic \"AI look.\"
So how can you use the latest AI tools without creating more AI slop?
Well, one way is to actually look at how AI
is right now used in the film industry.
I'm Tom Anthos, and I've been working as a visual effects supervisor
and also as a cinematographer and sometimes even as a director
on films, music videos, TV shows and commercials.
For the last 28 years, and I have used a lot of AI tools,
especially when working on visual effects, and a lot of this work,
you've seen it already on a big screen or Netflix shows and things like
that. You probably didn't even know I was using AI already.
I wish I could show you some of those bigger projects.
Unfortunately, I don't own the copyright to it.
But I did shoot a film almost eight years ago
that unfortunately never got finished because
We had major issues during the production of the film,
and ever since then I've been
trying to fix it in post-production using
traditional visual effects, which has just
taken me way too much time.
Thanks to Haxfield and their AI tools,
I've actually been able to make a lot of progress
on the film, so let's jump on the computer
so I can show you how the whole process works.
So here in DaVinci Resolve I have my short film project,
and here's basically a scene
that shows you a great example of how AI can be a lifesaver.
I'll let you kind of watch the scene.
Essentially, it's two girls on a boat trying
to push over this other girl, that's dead,
because there's a shark out in the water there, surrounding them.
And essentially, basically, they both end up in the water.
And that's what we had in the script,
and that's what had to take place.
Now the issue with this whole scene was,
this was actually like one of the, I think the
second last scene that we're shooting on our last day of filming.
And the actresses were all exhausted.
This was actually, the sun was disappearing.
I was exhausted, I was operating one of the cameras,
I was in the water for that whole day.
And so we essentially had minutes left to film this whole scene.
The shark actually in the film is a mix of stock footage,
well now it's also going to be AI.
We also did some shots where we did the 3D animated shark.
It's like a sort of a combination of effects.
Now for this part where they're supposed to fall off the boat,
the reason why it doesn't look very convincing right now is because
In the script, at least, it's said that the shark goes
and sort of bumps the side of the boat so hard
that the whole boat rocks violently left to right
and that causes the one girl to fall
and then as the boat swings the other way, the other girl falls off.
Well, as you can see, we weren't able to get that.
Now, part of this actually was also not even time, but this boat,
even though maybe it doesn't look that big,
it's actually a very heavy boat.
It's one of the old type of wooden boats.
And then on top of that,
we had our camera crew all on the other end of the boat,
you know, along with the engine and things like that.
So we tried, like the whole crew was rocking the boat.
We tried to move it as much as we could,
and we just could not actually rock it that hard.
So, you know, there's things that you sometimes imagine
when you're writing a script
and then, you know, things in reality don't always kind of work out.
So in this case, you know, I kind of just said,
okay, well, we're going to have to do it,
sort of fix it in post with the visual effects.
Well, again, it's very difficult
to fix some of these things with effects,
but now AI is where I would say actually
is a sort of a great resource
because you can create essentially
visual effects elements that you can composite using AI,
but you can even completely like recreate shots
that you weren't able to get,
like get extra coverage essentially of the scene.
So to start, for example, working on this scene, well,
first thing what you're going to do is you're going
to have to create elements that you can use.
So sort of like your elements to kind
of telling the AI what the location is.
And here, as you can see, I have a still shot
from earlier in the film where we just have the boat
and I actually painted out the actors that were in it.
So it's just the boat that's visible and then we can see the ocean.
So that's sort of our setting, our scene.
And then for our two actresses here,
I also needed to have these character sheets.
Now to actually create those character sheets,
everything is done right here in Krea AI.
So you'll actually go up here to Image,
and you can see you can have up here,
Soul ID, so Soul is the model that's being used in this case,
and you see ID Character.
And essentially what I did is I, you know,
click up here, Create Character,
you're going to load in your still photos of your actor.
Or, you know, references.
So in this case, I actually wanted these to be looking like
my actual actresses that are in the film.
So now, for example, if you wanted to create videos, as you can see,
if you just go hover over the video tab,
you see there's a lot of different ways
that you can sort of approach this.
Cinema studio is one way if you're actually working on films,
because it has already a lot of really great things in it.
But there's actually another tool
that I personally really love using and that's in here.
We just scroll all the way up.
You see up here you have Supercomputer. So I click on that.
And now in supercomputer, you know, you're going to start your project.
So I already started mine for Sharkbait.
But if you want, I can literally, like in here,
start typing in there some of my commands
and telling AI essentially what I'm trying to do.
So what this is, is it's basically a large language model, right?
AI, sort of like your assistant.
And it's sort of like this, I would say,
like a production assistant that will...
Get everything done for you. We'll arrange your crew,
we'll arrange your actors, you know,
all this stuff and, you know,
your visual effects department, make sure that
you're getting the kind of shots that you want.
So in the case of, for example, like in my film,
I wanted to essentially recreate that whole scene
with these two girls. Well, basically,
I start typing in there and giving it references.
So first, I'll actually kind of load in my
references. So I'll click Attach.
And I already have, as you can see, these references. So
I'll give it, for example, a reference for the shark,
so this could be the image.
A reference for how, like,
one of the shots looks, for example, like this,
because this actually has to match to the shot
or the scene before that,
so this is actually a still from that.
And then I actually have the ending one, which is just our dead body,
basically the dead girl that's on the boat at the end of the scene.
Then I'm going to load in my character references.
And once you have all of that selected, close it down,
and you can start basically talking here to your AI,
you know, filmmaking assistant.
And you kind of want to think of this almost
as if you were talking to like a real person.
So I'm just going to speed this process up
so you have to see me kind of type everything but
essentially I'm telling it that when I recreate
this film scene where with these two girls on
an old wooden boat in the middle of an ocean
and obviously I'm going to give it all references so
AI knows you know how everything's supposed to look
and then you see in here I'm telling it I
want to start with the underwater shot with the shark
and I want to give it a reference so I'm
Basically, the add sign and so I can actually select my image.
So in this case, it's my image one.
So essentially, as you can see, I'm telling it basically
that I want the first shot in this whole scene
to start with an underwater shot
with a shark that looks like it does in that image, right?
And then, you know, I'm going to tell it basically what the shark does.
And then again here, I'm going to give it a reference again, telling it
that I want to basically start looking like it does here in image two.
And then I can basically tell it specifically.
I want the girl on the left to look like this.
And then she lands in the ocean.
Okay, and then now I'll tell you guys,
once you have your whole prompt set already,
and again, think of it sort of like,
you don't have to be technical.
I mean, it can help if you know technical film language,
because then you can get more specific.
In this case, for example, like I'm just, you know,
telling it like basically the, you know,
kind of straightforward.
These are my references.
This is how I want the scene to look.
Now, if you know some film lingo,
like you know, for example, how to describe
what type of lens you want,
what kind of camera settings and stuff like that,
That always does help, but as you can see,
you can even just be sort of very non-technical,
just kind of as a creative,
kind of telling it the ideas that you want, what you want it to happen,
and you're giving it real-world references.
And that's where I think there's a really big difference,
is that if you just create everything with an AI,
that's where it basically tends to look like AI, like that AI slab.
Whereas when you're actually using real-world references,
then it actually looks more realistic, more like real life.
And then at the end, I'll show you kind
of what we do with the editing of all this,
which is sort of like another cherry on top.
But anyways, another sort of, I would say, big helpful tip is
to tell your AI assistant here kind of more
or less what kind of look do you want, right?
What kind of lighting, what kind of color grading?
So I'm just going to tell it
that pretty much what I have in my reference image here
of the two girls there on the boat.
That's kind of the colors that I want, because later on I'm going
to probably tweak the color grading in post-production,
but I'm going to do that to all of my footage, so I want those AI shots
to kind of match my sort of raw footage that comes out of the camera.
So in this case, I'm just going to tell it that...
I'm also going to tell it that I want it to be handheld
and kind of shaky camera moves.
Also, I'm going to tell it that I don't want any music.
Now another tip sort of what I learned
is that is very helpful again is to let your AI assistant
here tell them basically ask me any questions
in case there's something you're not clear about or
you need more information and then here I'm just going to click go
and we'll see kind of what happens
And yeah, what I really like about sort
of the way that Higgsfield kind of organized
the whole AI thing is that they really bring in like all the latest
and best AI models.
So you have all of these different models at your disposal.
But then they also sort of combine it
so that you can do everything in one sort
of place, you don't have to go
from one website to another
and that kind of thing. You can do it all here.
And that's where I think the supercomputer
is my favorite sort of part of it is because
It kind of takes and brings all the tools for you together.
So yeah, here it's asking me already, how long should the scene run?
I recommend about 20 seconds to make the shark impact and both
separate falls readable while leaving a brief final hold on the body.
So yeah, I think that that's okay.
So about maybe 20 seconds.
Um, maybe I can tell, or actually, you know,
because in my film that scene is right now about seven seconds,
but I don't mind extending it, so maybe I'll tell it 50 seconds or 15.
It says, your supplied character sheet
and bold images can anchor the scene directly.
Should I use these as is or create additional reference stills,
first for stronger visual continuity?
I'll use these 5 for the left woman, image 4 for the right woman,
and keep both barefoot to match
the boat scene rather than the sandals on their feet.
So you see, even in here what's cool is that it's kind
of noticed that there's a slight difference
because they were wearing sandals in my character sheets
and that's because, you know,
before they get on the boat they have those sandals on,
whereas in the actual scene up here they have lost them.
So that's where, yeah, you can actually tell it,
okay, feel free to create the updated sort of character sheets for it.
So you see if you didn't tell us basically to ask you
these questions ahead of time it would have
right away jumped to the sort of creating
this enhanced prompt for you but I find that that's
when it kind of some you know basically tends to create more mistakes
so it is always good to kind
of have the the supercomputer kind of kind
of clear certain things that it's not sure of.
But anyways, yeah, now you can see it's sort of ready to create
the thing so we can look over this and I would say you
Kind of want to get into a habit of actually reading these even
though some of these like look at this prompt
It's pretty long, but it gives you a lot of details
and sometimes you'll spot things that are like, oh man
Like no, I actually don't want to do this or I don't want to do that
So, you know you it is good idea to kind of go through it now
in this case just for the sake of this video
I'm not gonna kind of go through all this.
I'm gonna sort of trust that it's doing a good job
And I'm going to generate sort of a test of this.
Now, the way I'm going to do this is, you can select the model up here.
So you see, it's telling us right away
that it wants to select Kling 2.5.
I find that, yeah, I mean, usually you will select that.
It's one of the stronger models right now for video generation.
It's asking you the aspect ratio, which is,
you know, the same as matching original footage.
The sequence or duration, you can change this, obviously,
but, you know, we told it to do it for 15 seconds.
Now, here's the big thing I would say is whenever you're going
to generate your first sort of test of your scene,
you don't want to do it in 1080p quality because look at it,
it's going to cost you 240 credits.
And you will almost never be satisfied with the first generation
or with just one generation.
I, especially for these action scenes,
I like to sometimes generate like five, six or ten even
generations and I'll sometimes use one or two,
maybe combine them together, you know,
like shot from this one or that one in my editing.
But for example there's also going
to be a lot of stuff where it's just going
to generate things that are useless to me.
So in this case, it's actually better to go to 480p quality.
So you're essentially telling it that you want a lower quality,
but then afterwards you can uprise it because look at this at 480p,
at 15 seconds, it's costing us 45, right?
At 1080, it was 240.
So we can essentially generate a lot more of these
for the cost of one.
And you'll be able to see the action and everything.
And then you pick sort of your best take.
And you're going to just, for a little bit,
a few extra credits,
you're going to just up-res that to 1080p.
And it's all going to look good.
Versus if you did all of this for 480p,
that's when you're going to go through your credits
really quickly.
And that's when this AI stuff
can really end up costing you like thousands of dollars.
In the case of my work that I'm doing it like like
so far on this whole film
I've spent no more than like $200 on all my generations
and again that's because I'm being
kind of careful with you know what I'm telling it to generate
so but yeah otherwise you know
this is all pretty good here so I'm just gonna go and say approve.
And here we have our generation
finished so let's see essentially here
what we're getting so click on it
So I gotta say that you know the consistency
of the the scenes basically and the believability
in a way like I would say of the of how everything looks
is actually very good I mean like
it literally matches what we shot
and you know eight years ago and now obviously the big thing
for me is that this is all way too slow
so this scene has to be like a lot faster
and that's sort of the cool thing now about
this you know the AI computer you can kind of
Tell the supercomputer here, like, okay, I like everything,
but this has to be, like, way faster,
you know, make it seven seconds, so...
See I'm just basically telling it like what I want to change
and it's going to tell it and
and it's essentially what it's going to do is it's going
to rework the scene and they see right now
it's telling me okay I'm going
to do this this and that and it's enhancing the prompt now so that it
matches my new request closer.
And again I would say you kind of want to go through this you know
review it uh you know make sure again
that it's you're selecting the right ones in this case I'll
For ADP again because we're not sure how it's going to look.
And now look at it also the difference you know
because the the whole scene is shorter seven
seconds so it's 21 credits now.
Now in this scene where the girls are falling off the boat I actually
end up generating 10 different versions of this.
Now I only upscaled two of these to kind of
basically you know I picked my two best takes
and I kind of intercut the shots together. Now
this is how the scene just looks by itself it's
not color graded or anything like that.
But when I apply sort of a quick color grade,
don't judge this, this is not a finished film
and so the grading actually hasn't been done yet on the film
so that's going to be the last step.
But now you can see it kind of matches better
the live action footage before this and after.
And there's actually some live action
footage mixed into the actual scene itself.
And I bet you wouldn't be able to tell
which is basically AI and which is live action.
Thank you very much.
And that's because again I think it's better using AI sort of as a
tool to help you enhance your actual filmmaking.
Because the one big thing I found
is for example you know AI is great like you see
these kind of action shots
or when you for example need to do a special visual effect
that it just would take you too much time
or cost too much money to do it the traditional way.
Then again, AI can be great at it,
whether it's creating a whole shot, you know, as a visual effect,
or in some cases, for example,
just creating elements for a visual effect
that you're going to actually do yourself.
So that's where it's really good.
Where I still think AI is really, really bad at is the acting part,
and that's why I would say, you know,
at least for now, AI is not replacing actors,
and I don't really think it's ever going to replace actors.
And even when AI does ever get really close to real human,
basically, behavior,
I still don't think we really will want to enjoy
or really want to see, you know,
a robot perform or pretend like they're feeling something,
because they're, you know, obviously it's a robot.
Whereas human beings or really good actors
actually really get into the roles,
and they, at least for that moment on screen while they're performing,
they actually are emotionally in that headspace
and they're actually feeling it.
So those are sort of real genuine, you know,
performances or feelings that you're seeing the actors express.
Now, like I said, it doesn't mean that AI doesn't have a, you know,
its place in filmmaking, like I said, it is a great tool.
So I'll show you, for example,
like another shot that I worked on for this film.
It's actually this one here, as you can see,
is my whole chat with a supercomputer and creating these shots.
Essentially, I wanted to make it look like this girl
that's on the boat, she gets her head bitten off.
And I'll tell you actually, this is... this was really hard, like...
You'll see, this is just one chat where I
was doing this and I tried this multiple times
and you can see what it says up here, NSFW, so basically it's not safe.
And the reason why we'll give you sometimes that and the good thing
is as you can see it says credits have been refunded
But essentially we'll do that sometimes and that's sort
of the annoying thing with with AI or with certain models
Right now is that they have all these sort of barriers
and some of them I understand they don't want you to like steal
You know intellectual property or things like that like in this case.
Oh, it basically said it's too gory now in the end
I was able to still achieve the scene
I just have to sort of reword
and rephrase the prompt in order for the model to actually
accept it and do what I wanted it to do.
Now, sometimes you also get a message that says
that you cannot generate this shot for
you, and it's going to refine that to you.
But it says because it's infringing on copyright,
right, that it's stealing somebody's
intellectual property, which in this case,
I don't know how it came up to that conclusion
because it's, again, original film, original script,
original shots, actors, everything.
Again, the way to get around it is I just end up rewording my prompt
and it ended up working for me.
Oh yeah, and here, for example, I'm scrolling
in through all of these different shots
that I've been working on.
So as you can see, when you generate a shot in 480p,
like I said, to save credits,
at the end, to regenerate that same shot,
it would cost me an additional 112 credits.
So just click here and it will upscale
that and it actually looks really good.
I'll show you, for example, some of these other scenes that I got.
Oh, here's for example this...
shot that i was working on where
the shark grabs one of the characters in the film
and basically pulls them underwater
and i mean i i tried doing this as a 3d effect and i mean
it was just not looking good and it was taking
so much time and with ai i was able to get this
and you can see i got quite a few different variations of it
and i would say all of them
look pretty interesting you see this is the 480p version
this is the one that's in 1080p up here
And when you see it full screen, you really do see that it looks good.
But yeah, as you can see, if you're starting
with real, I think, footage
and you have actual real actors driving the performances,
then I think AI doesn't necessarily have to look that bad, right?
Now, there is actually another video that I've done before
where I used Higgs field to create a whole imaginary scene,
but using real world references so that, again,
we don't get that sort of typical AI look.
So if you like this video, definitely check out that one.
But also make sure you subscribe to this YouTube channel
or the newsletter on my website.
So you're notified when I release my next big project
because I've already shot another
film with real actors that is actually going to be all done using AI.
But the actors are actually going to be driving the performances.
So definitely going to be an interesting project
and I promise you it's not going to look like
AI slop.
Anyways, that's it for this video.
I'll see you guys next one.
Bye.