Today we’re going to take a quick look at matchmoving objects – but not by tracking multiple features and solving for the camera and object. Basically, not the normal way you would expect. This is how to hack and cheat matchmoving things that have no markers.
3D matchmoving is an absolutely essential aspect of visual effects . Ideally you’ll have some tracking markers and camera information, but if not, don’t fear – visual effects is all about cheating! And you can absolutely cheat an object matchmove. I’ve used this technique for years on TV shows like Gotham, Pennyworth, Agents of S.H.I.E.L.D., and Swamp Thing.
Here’s a couple examples of what we’re doing. These two videos each have slightly different effect elements added, but are both based off the same tracks.
For a real world example, in the show Agents of S.H.I.E.L.D., a character named Hellfire can charge things with energy and turn them into bombs. (I did not work on the shot below. Notice even his fingers have energy in them.)
You can also see it starting at 1:30 in this scene, and at about 17 seconds into this shot. And below is a still image I found online from one of the shots I worked on where Hellfire charges up a martini shaker.
More often than not, I don’t do anything to the default camera when I use this technique (other than roughly lining up the ground plane). However, you could be a bit more accurate by first using fSpy to calculate the camera position and focal length, and then using the manual techniques from there.
I created the electricity bolts for the bottle in Natron, and am including the Natron project in the Projects download if you want to take a look. I’d be happy to make a quick tutorial on it if anyone’s interested, just leave a comment below or message me HERE. (Edit: I went ahead and made the tutorial, it’s Part 4 below)
I’ve also demonstrated this technique a couple times during World Blender Meetup Day. In 2017 I did a quick walk-through of another Agents of S.H.I.E.L.D. shot, which you can see HERE, and then in 2019 I used the same footage I shot for this tutorial, which you can see HERE.
One of the most powerful things you can do with simple face tracking is blemish clean up and de-aging (making an actor appear younger or older). You could also add things like heat ripple, other kinds of particles, or cool sci-fi stuff like subdermal glowing veins or a HUD like in Iron Man. If your manual track is good enough, you could add makeup fx like the Joker, or even remove chunks of the person! Although, keep in mind that if you do try a digital makeup effect, manual tracking is not ideal for that. It’s more useful for things that don’t need to stick 100%.
Another thing I use this technique for a lot is gunshot woulds to the head, complete with chunks and blood mist blowing out. But that’s a tutorial for the future (I have big plans for that one!)
In the commercial software world, Nuke has a great plugin called GeoTracker that does an amazing job of automating this process. I also know of a tool in Syntheyes called GeoH that does the same thing, although I have never used that tool. There’s also a pretty cool tool called Character Face Gen. None of these options are free and open source, however, so hopefully this tutorial can fill the gap.
Actor – Stephan Singh
Assets required:
- Throwing_canteen_02_JPGs footage
- Throwing_canteen_02_CC01_PNGs color corrected footage
- Project files
- Texture elements
- Audio files
Pre-rendered elements:
- fractal_electricity_Natron_v01_JPGs
- bottle_matte_v01_JPGs
- burning_scorch_v01_JPGs
- burning_sizzle_v02_JPGs
- face_blood_v01_JPGs
- face_stitches_redness_v02_PNGs
- face_stitches_v02_PNGs
- smoke_v01_EXRs
- sparks_v02_JPGs
Tutorial 1 – Modeling & matchmoving the canteen.
Tutorial 2 – Matchmoving the face
Tutorial 3 – Adding the cool elements!
Tutorial 4 – Creating the electricity element in Natron
