HDV on Ice

Last issue, HD Focus discussed the usefulness of HD technology in helping the visual effects process on a 35mm-acquired feature film—the hit movie, 300. HD''s role as a useful effects'' tool appears to be a growing trend in Hollywood, and another recent film, Blades of Glory, also found a role for a flavor of HD in its visual effects work.

Blades of Glory incorporated three Sony HVR-Z1U 1080i HDV cameras into its facial capture methodology to match the faces of stars Will Ferrell, Jon Heder, Amy Poehler, and others to the bodies of stunt doubles doing complicated figure skating moves. According to Mark Breakspear of Rainmaker Digital, Vancouver, the visual effects supervisor on the project, the notion was to develop a sophisticated, detailed, and affordable facial capture system for a live-action project.

“The approach we came up with we call ‘P-Cap'' for ‘performance capture,'' ” says Breakspear. “The idea was to put our actor faces on the doubles as they did skating stunts. The typical approach of doing this is 2D method, where you film someone with a camera and take that plate and track it into the scene, but this wouldn''t have given the directors all the moves they needed. The moves in Blades of Glory were simply too complicated. These characters are doing big jumps, twists, and crazy physics defying moves, so we had to use stunt doubles, and we needed our lead actors'' faces on them. So we took the 3D approach, and basically built CG versions of the principal actors'' faces and necks.

“The process starts with filming a plate of the double doing the skating moves, and then shooting a separate element of the facial performance of the main actor. We needed to essentially record two things—the skin of the main actor, including the textures, and also exactly how the facial muscles move.

“To do that, we used a film camera in front of each actor's face with over 100 mo-cap markers in a mesh configuration over their faces. We shot with a 65mm lens to get a nice flat texture, and with some mirrors above and behind the head to reflect the various sides. That gave us essentially full coverage of the head from all angles. The material from the film camera acted as our basic texture. We lit it very flat, because we were going to light it all in CG, and all we needed was the rough texture of the face.

“But along with the film camera, we used the three HDV cameras, as well, to capture the actor''s face from three different, additional angles, with the mirrors again reflecting everything. We took that data and used a combination of [RealViz''s] Matchmover Pro and our own custom software to bring all those layers of data together to make a point cloud of dots. The HDV cameras were used to record the basic movement of the actor''s face, while the film camera captured the textures. That gave us more than enough detail to record the movement and match it to the CG models, which were built from scans of plaster casts of the actors'' faces. There was no way the production could justify using four film cameras for this purpose, but one film camera and three HDV cameras gave us everything we needed.”

Breakspear emphasizes this method allowed the production to capture more realistic detail than the kinds of facial mocap approaches used on recent animated films.

“The difference between this and what they did on [animated movies like Polar Express and Monster House] is, when they create a synthesized version of a human being, they don''t care if they don''t catch all the nuances of a human''s face—the twitch, the way the eye rolls, the way one eyelid moves slightly different from the other. They need a good level of detail, but they don''t need everything. In our case, we needed everything. We didn''t have huge resources, but we really wanted to capture what was intrinsically Will Ferrell or any of the other actors. This method meant that if you took the data we had from Will Farrell and added it to a different person''s face that person''s face would act and feel like Will Ferrell''s. This method lets you isolate the actor''s performance and put it into a CG face without losing any detail.

“We could not have done this any other way. The production could never have shot the hours we needed on film with four cameras. At the end of the day, it would have been too time consuming with film. Processing, scanning, storage costs, and everything else would have been elevated. But HD provided a cost-effective way to get the data quality without sacrificing what we wanted creatively.”

None of which is to say that Breakspear views HD cameras as a panacea for visual effects work. Quite the contrary?he maintains they “simply would not match up” to film cameras when it comes to shooting greenscreen, for example. But, as a niche tool to supplement the use of film, or “to help you with certain things after the main shooting process is over,” he considers HD an important visual effects tool.

“For instance, when you shoot lots of elements on film, like explosions, smoke, ice, other particles, and so on, and then go into post, inevitably there will be moments where there are decisions to make. Like when you see gaps in the people in a crowd scene in the background. So, you set up a bunch of people and capture them using HD, and then just patch up a gap, and it all works fine. HD has a nice ability to help with things like that, as opposed to shooting original elements. For that, I would always use film.”