James Knight, who worked on James Cameron’s Avatar while at Giant Studios, knows motion capture, or more precisely “performance capture.” It’s his specialty, as they say.
Knight is currently at BluStreak Media, a vertically integrated film and post studio, based on the Universal lot. The team has credits that include Real Steel, The Amazing Spider-Man (2012) and the Iron Man franchise. He also works with DuMonde Visual Effects, Richard Edlund’s company.
I thought it would be fun to get some insight and pointers from this LA-based Englishman on an area of the industry that not many know a lot about. He was kind enough to oblige.
So here’s James:
Performance Capture IS Animation
If you are a classic animator it tends to be like “jazz hands at 20 paces,” so some studios known for animation tend to be anti-performance capture. I remember when a BIG animation company came over to see Avatar; they really were only interested in the virtual camera and not the performance capture.
Not long ago, I gave a performance capture demo for BAFTA and the VES at USC’s Institute for Creative Technologies. Many animators showed up, and they asked a lot of great questions. Given that, and what I’ve been experiencing at work, more animators and studios are realizing performance capture goes hand in hand with animation. In truth, it’s a form of animation, and if anyone says it’s not an animation technique, they would be wrong. Example: Walt Disney used a form of motion capture as far back as Snow White in 1933. They rotoscoped actors to get the organic motion and the weight correct — you don’t hear of anyone not calling that animation, do you?
There needs to be an education of sorts because 100 percent of performance-capture data is touched by an animator. That data recorded on the stage, or volume as its called, gets you 80 percent of the way there. Performance capture gives you the real secondary motion and makes your character, or motion, more organic and more believable.
When you see a cartoon, music video, commercial or an animated movie, you want to suspend your disbelief that this isn’t real, right? Performance capture makes animation real. There’s an actor or animal driving that character — the rest has to be animated.
An animator has to “edit” the motion for a variety of reasons: Maybe a marker on the performer’s suit wasn’t seen by the camera. Or something moved too quickly and the system didn’t pick up on the marker’s location, and there was a pop — that needs to be edited/smoothed out by a human, a.k.a., an animator. A computer isn’t an artist. You need an animator to look at it… they are the artists.
Know the Different Types of Performance Capture
You can now use performance capture outdoors successfully, on the fly, or in your office, so it’s important to know the different types of performance capture available.
For films, music videos and TV, there are really only two different types: active and passive markers. There are marker-less systems, but those aren’t used for creating commercial content quite yet.
Active (or electronic, systems) are the kind that allow you to capture data outside. If you use passive markers (which are reflective), the sun is going to be hitting so many different surfaces the cameras won’t be able to tell what was a marker and what wasn’t.
PhaseSpace has an amazing active system whereby every marker on the suit has a unique frequency. This comes in very handy for real-time display of the CG characters on monitors around the set and in the post process, as there is no marker swapping. Basically, that means the system is never going to confuse your elbow with your wrist, for example. This removes the need to track markers in post, which speeds up the post process.
A recent example of active markers being used is in the film Dawn of the Planet of the Apes. The VFX company, New Zealand’s Weta, took the volume outdoors, and they used electronic markers.
The cameras, in this instance, knew to be looking for the pulsating markers and only see that. If they used passive, it would have picked up everything, and it would be a nightmare to track or get good motion from the performers playing the apes.
Actors Should Seek Out Performance Capture Work
Many actors in LA don’t know that you can audition for performance-capture jobs just as you would for a regular acting gig. I imagine they are less competitive jobs to get.
People are afraid that it’s so different, but it’s emoting… it’s performing. Many actors who haven’t worked with performance capture before are intimidated by the suit — but when they get in the volume they realize it’s just like being on stage.
In a sense it’s like going back to basic pretending, because they are interacting with these things that aren’t there — but they can see these things in real-time on monitors placed on the side of the stage.
It’s a great way to make a living for actors, if they go looking for the work.
Bonus (fun) tip: You can’t be modest because you’re going to be in a spandex suit.
Make Use of the Virtual Camera
Part of motion capture, rather than performance capture, is the virtual camera for animators, directors or a DPs. The virtual camera is a great viewport into the virtual world. It’s the director’s or the DP’s view, and therefore the audience’s view.
Imagine an iPad or tablet with gaming joysticks on each side… that’s what it looks like. It has markers on it, which are picked up by the motion capture cameras, and the motion of those markers drive the view of the virtual world, which is simultaneously transmitted to virtual camera you’re holding. Make sense?
By using a piece of software called Autodesk MotionBuilder you can record your virtual camera movements. This draws the audience in even more. That’s your end user. If you use a virtual camera — even if you don’t use performance capture in your stuff — you can still benefit from its use.
Virtual camera is largely unknown to the masses, and I think it’s a tool people should be using more often. And if you go searching it’s relatively cheap these days.
Another Bonus (fun) Tip: If you are going to be working in a performance capture suit, make sure you shower.
Be Careful What You Wear
Pay attention to what you wear to a performance capture production. If you are going to be working on a project with performance capture and are going to a facility to do it, don’t wear shiny shoes or white t-shirt because the cameras will pick up the reflections.
If you are using an Active system, like PhaseSpace, it won’t matter, but with passive volumes be careful. Things you wouldn’t think of, like the Nike Swoosh or the Adidas stripes … we’ve had to put duct tape over those things to make sure they wouldn’t reflect because that messes up the data.
I Leave You With This
Learning more about performance capture is great for a film student, animator or anyone looking to break into the business. The entertainment industry is hard to get into, and even when you do it’s also hard to make a sustainable living, But there currently aren’t thousands of performance capture specialists out there, so look into it at a local college or university, or intern at a performance capture studio!
Three people I worked with on Avatar did just that, and all have great careers and have worked on some impressive projects.