There’s times that replacing a screen is necessary because you either don’t have the visuals ready before filming, or coordinating the movement of everything is too complicated. Or maybe you just can’t get it to look good filming on a screen on location. Whatever the case, you’ll need to place your visuals onto your screen in post. But how do you do that? We’ll lucky for you, we’re gonna show you how to pull off this effect in After Effects!
Replacing Your Screen In After Effects
This text has been transcribed from the video for optimal reading
So here we are inside of After Effects, and the first thing that we’re going to do bring in the footage that we want to work with. Our example here is a shot from Motion Array’s Library and you can find the link in the description below. It’s a person with their tablet on a table and they’re swiping through some imaginary images.
Step one is simple, take your footage and trim it down to only what you’re going to be using. We’re going to be adding some effects that will make After Effects work pretty hard, so the less unnecessary frames you have to render the better. If you’re not sure if your decision on how much footage you’ll use will work, just use the B and N keys to set in and out points and work with that. So for us our footage looks like this. Now what we’re going to do is create a solid that will double as the screen so that we can line everything up nicely to start with.
Create a new composition by going to composition, new, composition. This will act as the screen of the tablet. I went ahead and looked up the dimensions of the tablet used in the shot, but really even just an approximation will do if you’re not 100% sure. So i found that the dimensions are 1024x768. But it’s vertical so it’s actually 768x1024. You might also have to uncheck the lock aspect ratio to make this work. Name it so that you’ll remember it’s the screen composition and Hit okay.
Cool, now this is what will appear in your screen. Now we’re going to add a solid that will contrast the green easily. So i’m going to click the swatch here and choose red. Go back to your main composition and now let’s drag your screen composition into your main project.
Next we’re going to line up this screen composition so that it matches the actual screen of the tablet. we need to add a Corner Pin effect. Go to your effects panel and search for it, and then drag and drop it onto your composition layer. When it’s highlighted, you should be able to see dots on the corners of your layer. If not, make sure layer controls are checked by going to view > show layer controls.
Drag Corners really rough at first and then when it’s closer do a really fine tune job. And it’ll help to drop the opacity of your screen composition so that you can see both at the same time. Get as close as you can, then i’d suggest to scale up ever so slightly so that it’s just a tad larger than the actual screen. A little too big is better than too small.
Bring opacity back up and then this is what you have. Great. Now go into your composition for your screen and this is where you decide what’s actually in your screen. For me its a pic of a dog. Drag it in above your red layer, and just hide your red layer instead of deleting it in case you need to re-adjust things later. You can also add video if you’d like, and it’ll work the same. But for simplicity and because our example is portrait I’ll work with this still image for now.
But when we go forward here we can see we have a problem. Our finger goes beneath the picture of the dog. So there’s two main methods to fix this. One is you can keep the photo on top and rotoscope the finger to out of each frame individually. That’s not the method we’re using, but if you want a quick touch up on rotoscoping, check out this video we did here, and the link is right here for you to check out. If you use that method, you can join back in once you finish rotoscoping, but for us in this example, what we’re going to do is take advantage of the fact that the tablet has a green screen, and we’re literally going to work with it as if it was a green screen. So drop the screen comp below your footage and now let’s search for the keylight effect. And actually in just one click you can see that it does not too bad of a job. But there’s a couple problems we need to address.
First, our green didn’t entirely key out as there were some strange glare and banding from left over. And then there’s also some blur on the finger as it moves through the image. So let’s take care of both of those. Go to the view and set it to screen matte. Now you should see a black and white representation. Black is transparent and white is opaque. You want to focus on the screen for now and make the screen completely black as possible. So go down to the screen matte options and move the clip black slider until its completely black and no banding. Cool. Now take the whites down until it becomes a lot better too here outside the screen. In this case you don’t have to worry about the outside too too much because we’re going to be isolating the screen in a second. But in other cases it’s good to get everything as black and white as possible.
Great. Now let’s go to final result and we can see that it’s looking better, but it’s still not amazing here with the finger. There’s some discoloration when it gets blurry moving from right to left. So let’s go into another view mode. Status. And here we can see that we actually didn’t get all the screen completely keyed out. So let’s keep raising the blacks and get our screen completely black. Then we can notice that our finger has a grey fringe around it when it’s wiping across. This is that motion blur that makes it really tough to distinguish for keylight. So we’re going to keep playing with the clip black and clip white until we get it to a place where we reduce the grey around the finger but we don’t take away the actual realistic blur. It’s a happy medium and it’s going to take a little trial and error, but if you’re looking at my clip as a reference, I found that clip black of 64 and clip white of 92 work really nicely.
Great! And lastly what I’m going to suggest is to take your locked despill bias and alpha bias and select a color that’s in the actual image you’re projecting onto the screen.
And this is our final result. It’s really looking great! But there’s one other thing that it looks like we missed. In taking away the green from our screen, we also took it out of the plant here and the pens. So how to we isolate this to the screen only?
Let’s make a mask by highlighting the actual footage layer, and hitting q to bring up our rectangle tool. Then click and drag over the screen. It doesn’t have to be perfect. Just so that it captures all of the screen and excludes the other green elements. Now go down to your mask in your layer and set it to none. So now we’ve created a mask that’s not active, But we can tell only keylight to use it by going to inside mask, choosing our mask 1, and then hitting invert. Now our green outside of the mask is returned and we still key out our image!
And guys this is our effect! But let’s take it just one quick step further by animating our picture off screen. Go to the first point of contact that the finger makes here. Now dive into your screen comp. Your playhead should be at the same place as in your previous composition. So set a keyframe for position then go back. Now move forwards to the place where you want your image to be fully off screen. And dive back into your screen comp and keyframe your image offscreen. Nice
Now let’s see what it looks like. Set a beginning and end point with B and N respectively to watch your footage on a loop. It’s a start, but we need to make it more realistic. Let’s go back to our keyframes and highlight them and add an easy ease. It’s getting better. And if you really wanted to finess it, you can higfhlight your keyframes, and open up the graph editor. From here you can tweak your motion to perfection. But let’s add the finishing touch by going to our motion blur option and selecting it, then enabling it for that image layer in specific. And if we place in another image for realism, like that person is scrolling through images, this is what we get. And this is what we get for our final result.
But if you study this image in depth, you can pick out that it’s a little too perfect to be projected from a screen being recorded. In a pinch, this will be good enough, but let me show you two more things that will really help to sell the effect.
First, add a realistic blur by searching for camera lens blur. Add it to your footage, and blur your footage out really really subtly. Like only 1-2 pixels of blur. If you don’t do this, its assuming that your screen was in perfect focus on an amazingly expensive lens.
The second piece might also not be as intuitive. Go to your comp for your screen and add a white solid on top. Then set a mask for the solid that’s going to represent some fake screen glare from an off camera light source, cutting across only part of the screen. Now add a subtle feather to it and drop the opacity down really low, like below 25%. And now when we take a look at the final image we can see that there’s a little more of a realistic look to our screen. It’s subtle but it makes the whole thing feel a little bit more reel. And guys that’s how your replace a screen that’s not moving. But what if your screen is moving. Let’s dive into a second example to check that out.
Okay so here we have a woman who’s on her phone and it’s moving around in her hand. To actually make any sort of screen replacement work, we need to track the movement of the screen exactly. Now, if you can see all the corners of your screen, a traditional perspective corner pin track could work. And we did a video on tracking objects using after effects built in tracker and the link to that video is right here if you want to learn more about it.
But when we tried doing a corner pin track on our footage, we noticed that it was very easily disturbed by the woman’s hand here interacting with the phone. So instead, we’re going to be using an incredibly powerful tracker that comes with After Effects. It’s called Mocha, and we’re going to walk through together how to use it. It’s actually easier than you’d expect, so let’s go through it together.
With your footage on your timeline, go up to animation, and track in mocha AE. And Mocha should recognize the footage and automatically set up your composition for you. But there is the rare occasion that it doesn’t recognize when you bring it in. If that’s the case, what’s probably happening is that your footage format isn’t recognized. So an easy solution is to just take your footage as it is, and export it as an H.264 format. That’s one that mocha will read easily. Then bring it back onto your timeline and keep going with our next steps.
With your footage on the new mocha timeline here, we’re just going to do a couple simple steps. Bring your footage playhead back to the start. Click the x-spline tool and now like the pen tool in after effects. Click and make a basic shape around the screen of your phone. As long as the phone is within the confines of your mask, it’s pretty good at doing the rest of the work for you. You’re just telling Mocha where to look and the rough shape of your object. Once you’ve finished your last dot, hit the right mouse button to complete your outline.
Great, almost done, but before we track, we’re just going to make things a little easier for After Effects later on. Go to this tool called the show planer surface, and click it. Now a box comes up and you’re going to use this to tell Mocha exactly where the screen is on your phone. Just click and drag these corners here and try to get it as exact as possible. Up in the top left corner you should get a zoomed in view to help you be extra accurate. This last corner we just have to estimate by using the parallel lines here and get the borders as close as possible to the existing boarders of the phone.
Great! Once you’re done, go down to this track forward button and click it to let Mocha do the work for you! It’ll take a little while, so you can take a break and come back in a few minutes. But wat you should notice is that even interractions like the hand moving across the mask won’t interrupt it at all. Becuase you’ve specified the overall shape of the phone, and the planer surface of the screen, the track is going to be incredibly precise and it should stick very well.
Now that that’s done, you just need to tell after effects to use the data that mocha just collected. To do that, we’re basically going to copy and paste our findings into after effects. Go to export tracking data, and choose “after effects corner blur supporting motion blur”, and then when you hit copy to clipboard, you’ve basically just copied a bunch of information and the next time you paste, it will paste all of the information Mocha collected. So go back to after effects, and let’s just quickly set up a new Solid to double as the screen. Right Click here and go to new, solid.
Let’s make it yellow so that it’s really easy to contrast with the blue of the phone. Cool. Now let’s pre-compose it and label it so we know it’s our screen composition. Now we need to bring our playhead all the way back to the start. This is because the mocha data was measured from the first frame. Now when we go to our transform properties of our screen composition, highlight it, and hit control or command V to paste. All our tracking data should now be pasted to the composition with the yellow solid. And you can play your footage back to see that it’s sticking incredibly well! It’s lined up perfectly to our phone screen.
And guys that’s honestly the only part that’s different from our previous still example. From here on out we’re just going to blaze through the same process that we did for the still tablet because we don’t have to worry about the jiggling motion now.
Let’s add in our video here in our screen layer. Drop the screen layer below our footage layer. Add a keylight effect. Key out the blue. Mask around the screen. Set the mask to keylight. Tweak the blue key. And animate our footage to be a little bit more engaging, making sure to enable motion blur. And top it all off with a lens blur a little bit of fake screen glare.
Well, that concludes this tutorial. I hope you found it helpful, if you did, please give us a thumbs up and if you’d like to see more tutorials, we’ve got lots of other tutorials for Premiere Pro, After Effects, and filmmaking in general!
Thanks for watching and see you in the next video.