Animated Performance

 Web Expanded
Step by Step: Beowulf

Advanced fire simulation was essential for illuminating the motion-captured world of Beowulf.

“A lot of technology had to be created in advance.” This comment from Jerome Chen, the senior visual effects supervisor on Paramount Pictures' Beowulf, is the ultimate understatement. The digital epic, directed by Robert Zemeckis, pushed Sony Pictures Imageworks technology far beyond what had been done for Zemeckis' previous performance-capture productions, The Polar Express and Monster House. Unlike those fancifully animated family films, Beowulf called for mayhem to be visualized at an unprecedented level of realism. And that meant advancing the technology for both facial performance capture and the simulation of several kinds of effects.

Chen, who had worked with Zemeckis on The Polar Express, says Beowulf had more of a live-action aesthetic. “In The Polar Express, we did a lot of cutting and pasting of motion-captured performances, and that turned out to be more distracting than helpful,” Chen says. To some degree, that was unavoidable, he admits, because actor Tom Hanks had to be motion-captured playing multiple roles. “But for Beowulf, Bob decided we'd go on set and capture the actors' motions and not go on to the next setup until that scene was correct. He wanted to avoid cutting and pasting.”

Once the cloud of data points that described each actor's performance were applied to digital puppets and given to Zemeckis, the director then freely chose his various camera angles for the scene. Chen says he thinks such freedom is one of the biggest advantages of such a modular production. “Bob could focus on each step of filmmaking discretely,” he says.

Refinements in muscle-simulation technology were crucial to the animation of a believable warrior.

For the actors, whose performances were captured on a 25'×25' stage, the mo-cap recording felt like a stageplay. “I heard both Anthony Hopkins and Angelina Jolie mention that they didn't have to worry about playing to a particular camera angle, but could focus on the other actors. It was shot in full scenes, and they went very fast,” says Animation Supervisor Kenn McDonald.

A key advance that Imageworks made with Beowulf was the simultaneous capture of facial performances. Each actor had an HD camera dedicated to getting close-up facial-reference footage, but tracking markers also captured the subtleties of face and even eye movements. “We found a technology used by ophthalmologists called EOG — electric ocular graphography. Electrodes measure the electrical impulses that move the eyes, and we had them wired into mini-pocket computers. This data contained almost subliminal nuances, and we captured that at the same time we were capturing the performances of the actors' faces and bodies, so we could sync them up,” McDonald says.

The rigs used for facial capture were also customized for each actor's performance. “Some actors could raise their brow higher than others, and their cheeks bunched differently. Every facial rig ended up being specific to that actor,” McDonald says. “So if Anthony Hopkins comes back, we're ready to go!”

“On Monster House, we had the introduction of a pose-based system called FACS [Facial Action Coding System],” McDonald says. “First you have an actor go through a series of expressions and poses and phonemes, and then you recreate those expressions and phonemes on the digital character using a facial rig. You take the performance-capture data for an actor, and using a matrix of all these expressions, you do a solve that basically breaks down any given frame to a combination of expressions and phonemes. That would give us an initial impression of each performance. Then the animator, using a set of controls laid over the top of that performance-capture data, could finesse the expressions to make sure we're nailing the intent of an actor's performance.”

“I don't like to use the word ‘realistic'' because obviously it's not real,” McDonald says. “We're not trying to convince people that's actually Anthony Hopkins — it's a version of him. We tried to capture an authentic performance that's accurate to his original intent, with a high level of detail. If Anthony had only recorded a voice and we were crafting the visual performance based on a limited amount of video reference taken during a voice session, we'd have 20 animators trying to maintain a consistent performance. By having a single actor drive the performance, the little twitches he used to create his character will be consistent throughout.”

“We've gotten better at sorting through the tracking dots and finding the nuggets that we need to apply to a character. But everything you see in Beowulf is an actor's performance. Mo-cap movies are actor-centric, and they'll continue to be,” Chen says.

Once the performance data was tracked in MotionBuilder, animation began in Autodesk Maya. Imageworks also applied its newly refined muscle simulation system. “Beowulf likes to take his clothes off, and he runs around naked for a lot of the movie. Our system allowed us to simulate muscle jiggling under skin. It required an iterative process to achieve the authenticity that we were shooting for,” McDonald says.

Simultaneous facial and body capture helped produce integrated mo-cap performances for Ray Winstone, Anthony Hopkins (top) and Angelina Jolie (bottom).

If the devil is in the details, Imageworks animators and texture artists had several demons to wrangle. “Creatively, we had to reach a level of detail where we could make the world of Beowulf feel believable enough so you could watch the story,” Chen says. “We're not trying to fool people into believing this was photographed. It's a fantasy world. We thought we'd stop much earlier in terms of stylizing it. But it looked like the skin needed to be more realistic. We also got to the point where we wanted to see hair coming out of King Beowulf's ears to make him more interesting.”

Simulated hair and clothing were handled with Maya, but atmospheric effects such as dust, smoke, and snow were done with Side Effects Houdini, and this challenged Imageworks to develop a hybrid pipeline that included significant proprietary software. Nowhere were these challenges more evident than in the simulation of fire, which is plentiful throughout Beowulf. “Everything in this movie is lit by fire,” says Sony visual effects lead Vincent Serritella, who co-developed the fire simulation used on the film. “This story takes place before electricity, so we had hundreds of fire sims of candles, torches, and fire pits.”

The original development on this system had been done for Imageworks effects in Ghost Rider, which had a character made of fire. “That was supernatural fire,” Serritella says, “but we've been able to leverage 75 percent of that pipeline. For Beowulf, we wanted the natural integration of how fire transitions to a carbon, where fire lights the smoke and the smoke blows around. Now the fire sits in space when it lights the smoke. And we took our fire rendering a lot further, using a volumetric rendering approach rather than the [Pixar] RenderMan approach that we did on Ghost Rider.”

Imageworks began by importing matchmove or environment data into Houdini, and the team did what Serritella calls “field building.” The graphic artists would literally paint the places they wanted fuels for the fire, indicating which areas they wanted to burn and where they wanted collisions to happen. “If we wanted fire to wrap around a character, we'd have the character itself be a collision object. Then we'd write that data out to disk and use Maya's fluid solver engine,” Serritella says.

The Imageworks fire-simulation system uses concept of simulation boxes, each of which has parameters that a particular piece of fire is simulated within. This method gave Chen and Zemeckis the ability to specify if they wanted ambient wind affecting the fire, for example. “If they wanted wind coming from the left, everyone working with smoke, fire, and embers would have their wind direction pointed that way. We could make a wind tunnel for our boxes and add turbulence to the sim,” Serritella says.

“Each piece of fire had a voxel space around it, basically little 3D pixels,” Serritella says. “If you have a tight, conservative box, you'd clip the fire and it would stop sharply along an invisible wall. So you give enough padding to let your temperature heat up and then fade appropriately. We'd strategically get the lay of the land for a shot, map out what's going to be on fire, and, based on our boxes, set up an automated process to dice up the scene.” This helped optimize the rendering process, which was done in Svea, Imageworks' proprietary volumetric renderer.

“Beowulf pushed the scale of using a volumetric renderer. If you have a volumetric solution, it's a true 3D render of smoke. So you could rotate a camera around it, and it will look like a 3D object. If you have lighting that rakes across it, you get to feel the parallax of the light as the camera moves around it,” Serritella says. This approach gave Imageworks compositors a lot of control. “If they wanted to do a pseudo-camera-depth blur in the background, they could do that. It really was a playground of effects.”

So is Beowulf, in the end, an animated movie or an effects movie?

Jerome Chen has a simple answer. “It's all these things. Why does one have to preclude the other?”