9 EPIC Generative Fill Techniques for VIDEO!

In this video, I'm going to show you nine ways to upgrade your video projects using Photoshop's Generative Fill. And as a bonus, I'll also reveal a few tricks to help sell the effect and make everything look less fake. Let's get started. You can use Generative Fill to completely transform the environment of your shot. It's important to make sure that there is an element in your real video that fits your fake environment. In this case, the grassy trail. To get the best results, it's recommended to use footage that was shot on a tripod without moving the camera. But by the end of the video, I'll also show you a more expert technique when you're dealing with moving footage. In this shot, I'm keeping the character walking on the trail. I'll export a still frame, save it as a PNG file, and open that file in Photoshop. I'll make a selection of the character and the trail, and then invert the selection by hitting this button. So now everything around the character and the trail is selected. And this is where the fun begins and you can let your imagination flow. In the prompt box, you can describe what you want the environment to be. Perhaps something like Swiss mountain landscape, and then hit generate. Let's have a look through the options and I like this option right here. Next, I'll turn off the background layer that has the character and the trail, leaving me with only the generated image and a transparent area where the trail was. I'll go to file, save as, and then make sure to select PNG as file format. Now I can import this PNG file into my video editor and place it on top of my original footage. Then I turn both clips into a compound clip, add some color grading, and we have successfully teleported to a different location. This also works really well with top-down drone shots. Make sure to add some fitting sound effects to really sell the idea that what your audience is seeing is real. We can take this even further and incorporate our original footage in a completely new, imaginary, and much larger environment. I want to turn this plain garage shot in a more exciting science fiction shot. I'll just open a still frame from the footage in Photoshop and change the canvas size, making it larger. Next, let's start building a scene around our original footage. I always leave a bit of overlap when making a new selection. This makes sure that the generated image blends in better. You don't always have to put in a prompt. You can also just see what Photoshop's AI mastermind comes up with and leave the prompt box empty and hit generate. So that's looking pretty good, but I want to add a few elements in the foreground that will help create more depth to the shot when I later add some movement to the whole thing. And one last thing to really sell the idea that this is a real sci-fi set, let's add some glow to these lights here and here. Afterwards in the video editor, we can create a very simple animation for these lights. So in a new layer, I'll pick the color of the light and just add one brush stroke and then add another layer for this light here and another brush stroke. I'll set the blend mode of these layers to overlay so you have a better idea of what this is going to look like later on. All right, now that we have everything that we need, I will first go to the layers that have the generated foreground elements, merge those two layers, make a selection of those elements and make sure to feather the selection. Then I'll duplicate the layer. That's going to leave me with a more feathered version of those elements. Next, I'm going to turn off all the layers so that I just have the foreground elements on a transparent background and export this as a PNG. Then I will take my background layer without the foreground elements and without the glow and export that as well. And lastly, turn off all the layers except the ones where I painted that glow for the lights and export those as separate PNG files. Okay, now let's bring everything into the video editor. So first the background image on top of the original video, then adjust the size of the original video a bit. And if I just play this back, you can tell it already works pretty well. You could actually be done right there, but we can add a bit of spice and just make the whole thing more credible by adding some camera movement, those foreground elements, and then animate the lights. It's super easy, but very effective. All right, let's start with the lights. I will add the PNG files with the lights on top of everything and then change the composite mode to overlay. That will create the same glowy effect like we saw in Photoshop. Now I will just add a fade in and a fade out to the clips and let's play that back. You can see it looks like those lights are kind of pulsating. Okay, next let's add the camera movement. I will create a compound clip of the original video, the background layer, and the lights, and then make a digital zoom out by adding a key frame at the start and increasing the size, then adding another key frame at the end of the clip and decreasing the size. Next, I will add my foreground elements and again, add a key frame at the start and increase the size. But because this object is much closer to the camera, so moving a bit faster, I will increase the size a bit more. And at the end of the clip, I will add another key frame and decrease the size. All right, let's play it back with some sound effects and color grading added. If you're watching this video, chances are you like to learn. So let me introduce you to Skillshare who are very kindly sponsoring this video. Now you've probably heard a thing or two about Skillshare by now. For example, that they are a fantastic learning platform where you'll find online courses on almost anything, whether you want to improve your Photoshop skills, get better in DaVinci Resolve, redesign your home, learn watercolor painting, or become a master of productivity. I've personally really enjoyed diving into some of the chat GPT and other AI courses. And I love the cinematic lighting course by Zach Mulligan, a cinematographer who's worked on HBO, CBS, and Netflix shows. Of course, as a YouTuber, I got a lot of value from the YouTube classes by Ali Abdaal and MKBHD. But let's say you're looking for ways to up level in your career or start a new side hustle. Well, Skillshare offers creative career focused classes where you can learn more about things such as entrepreneurship, freelancing, marketing, or social media. So whatever it is you're interested in or want to learn, Skillshare has got you covered. And the best thing is you can now check out all these classes completely for free during the one month free trial. If you're one of the first 500 people to hit the link in the video description, you can sign up for a completely free 30 day trial of Skillshare. And you can get 40% off your first year of Skillshare membership. So check out the link in the video description and thanks to Skillshare for sponsoring this video. Now let's get back to some generative fill magic. And while we're on the topic of changing your environment, you can even use generative fill to create an entire range of different sets, add different elements, and of course, change the size of it. And all you need to do is film yourself in front of a blank wall. Then bring that still frame into Photoshop and let's start by expanding it a bit. I will select everything but me and simply ask for a room. I like this room, but not this window, so I'll change that. And now I'll just start adding elements to the room. It takes a bit of time and a few tries to get what you want, but eventually you'll get there. And just like that, your pretty boring talking head footage can be turned into a much more interesting shot. But here's a cool trick for you. You see this window here, it looks kind of boring just like this, right? So let's fix that. I've placed this stock video from some trees on top of everything else. I'll adjust the size a bit. And next I will mask out the window so that the video with the trees is behind the window. And this will be more or less complicated depending on what the window looks like. I'll also add some blur and change the exposure a bit. And having that subtle movement behind the window there makes a much more realistic impression. You can also use generative fill to remove objects from your video footage. Just take a still frame into Photoshop and make a selection of the object you want to remove. Then hit generate without entering a prompt and bring the file back into your video editor. Let's play it back. And voila, you have created a cool effect right up until you start walking through the shot and the whole thing gets ruined. But we can easily fix this with some simple tracking in for example, DaVinci Resolve. You can of course also use another software that has masking and tracking abilities. I just duplicate my original video and then place it at the top of the timeline. Then I go to the color page and to magic mask, add a new node and add an alpha output. I'll make a selection of the character and if needed, make some adjustments to refine the mask. Then let DaVinci do the tracking. And what we are left with is Photoshop AI meets DaVinci AI video magic. Generative fill is also great to clean up your video because you might have a perfect looking shot but there's something distracting in your frame. Like in this shot, I don't like all the houses in the background here and here. So I'll just take that frame into Photoshop and select the areas that are not working in the shot. Then hit generate to replace it with something that looks better and makes the shot look less messy. Next, I'll bring the PNG file back into my video editor like we've done before. And now that shot looks so much more satisfying. You can also use this technique to get creative with lighting and not have to worry about getting those lights in your frame. You can remove the lights in Photoshop and then add that PNG on top of your video footage. This way you'll end up with unique lighting in a clean looking shot. And there's really no limit to what you can do when combining generative fill and video editing. You have to be some kind of VFX master sensei to create certain effects. But now to add imaginary objects to your video, all you need to do is make a selection where you want the object to be, write a prompt and hit generate. To really help sell the effect, you can add a nice color grade, some camera shake and a lens flare, a few sound effects, maybe a green screen spaceship, and you have the ability to completely change reality. Besides creating new objects in your video, generative fill also gives you the ability to transform existing objects. You simply make a selection of the part of the object you want to change, enter your prompt and click generate. The possibilities are endless. And similar to what I've shown you before, you can even change things behind moving subjects. In this shot, I want to turn these trees into some palm trees. This looks pretty good, but as you can tell, as soon as the car drives through the scene, the PNG with the generated palm trees is covering the car. To fix this, I'll duplicate my original video track and place it on top of the other ones. Go to magic mask in DaVinci Resolve, add an alpha output and connect the node. I'm just going to make sure that I have a frame where the whole car is visible. Then I will select the car and have magic mask track the car. After a few slight adjustments to the magic mask settings, I end up with a pretty smooth looking shot. All right, so far we've only been working with locked off shots where the camera wasn't moving. But what if I told you you can also use generative fill with shots where the camera was moving? Let me show you. I have this cool looking shot that I got off pexels.com and as you can tell, the camera is panning towards the right. So I just took a still frame from this shot and generated a few objects in Photoshop like I've shown you before. I did make sure to refine the selection of those objects. Of course, if I play this back now, you can tell those objects are stationary and the shot is not, so let's fix that. I'm just going to remove these video layers here and then go to the fusion page in DaVinci Resolve. You can also do this in other software that has tracking features, like for example, After Effects. Then with the media in one node selected, I'll hit shift space to bring up the tools and select the tracker. I'll select a point to track, something with high contrast usually works best. And then hit this button here to have DaVinci track the shot. Once it's done, I will add my first object, the tower to the nodes and connect it to the tracker. Then select the tracker, go to operation and change it to match move. You'll see that the object is too big right now. So with the object node selected, I will add a transform node. Then select the frame where I generated the image and match the size and the location of the tower. The green frame helps with matching both images. All right, next I'll do the same thing for the second image, the spaceship. So add another tracker, make sure that I'm at the start of the clip and then select a point to track and make all the adjustments. And let's play it back. Looks pretty good for some quick tracking. So what do you guys think? Is Photoshop generative fill something you might be adding to your video projects? Of course, we're only at the early stages of what this AI technology can do and Adobe is constantly innovating. So who knows what's possible in a few years or even in a few months from now. Anyway, that's it for me. Thanks a lot for watching and I'll see you in the next one, bye.