Building Character

Animators at Pacific Data Images (PDI), Palo Alto, found themselves struggling late last year to keyframe moves for the villain from the upcoming PDI/Dreamworks CG feature film, Shrek. Eventually, it became clear to filmmakers why their initial attempts to bring the evil Lord Farquaad to life were failing.

“Farquaad is a tiny man with a near photo-real looking face and a distorted body,” explains Raman Hui, Shrek's supervising animator. “We initially had him moving in a theatrical way, very exaggerated. Eventually, as we began matching the voice track with the animation, we realized we weren't completely on the right track. The character is voiced by John Lithgow, with a booming, powerful voice that didn't seem right coming out of this tiny character. We eventually decided he needed more subtle breathing movements, chest expansions. Therefore, we built special chest heaves into the animation controls, and whenever we heard a sound that suggested breathing on the audio track, we inserted one of those chest heaves.”

Hui and Andrew Adamson, the film's codirector along with Victoria Jenson, say PDI also added more head and neck movements to sell Farquaad. “The idea was that, for Farquaad to project that kind of voice, he needed to put his whole body into it and labor with his breathing,” explains Adamson. “That change made a huge difference.”

How to achieve “believability” with CG characters has been debated for years. With such characters now dotting the media landscape, filmmakers and animators have made significant strides. But like Lord Farquaad himself, believability is still a subtle thing that is difficult to define, let alone execute.

Animators, directors, and producers currently working on major CG, character-oriented projects recently highlighted for VisFX some of the major developments that have helped raise CG characters out of their infancy and into adolescence.

In the CG world, a character's initial design impacts what new tools filmmakers will essentially have to invent or improve as they move into production. If details such as hair, fur, cloth, eyes, and skin, among others, are not defined early, it may be too late to create tools to handle them later.

“We initiate a significant R&D project at the same time we initiate design work in the earliest stages of preproduction,” explains Ed Catmull, president of Pixar Studios, Emeryville, California, and recent technical Oscar recipient for his work on RenderMan's texture-mapping functions. Pixar, now currently deep into production on Monsters Inc. (due for a November release), is already involved in R&D for two more upcoming features.

“Once you are in production, certain character components have to be locked down,” says Catmull. “In Monsters Inc., for instance, much of our innovative work was done with hair and fur because ugly, hairy monsters star in the film. One character, though, wears a shirt, which meant we had to improve our cloth tools to work smoothly on the body of a monster. We had to turn that specific, creative need into a production tool early on. For our next film after this one, Finding Nemo, there is an entirely different R&D project under way.”

Similarly, a few years ago, Disney Animation itself initiated a major R&D project to improve the studio's ability to manipulate muscles under, and in concert with, flesh and skin. That fruits of that project were featured prominently during last year's animated feature, Dinosaur.

DreamWorks/PDI and Disney/Pixar are no longer alone in such efforts. Years of R&D likewise preceded current production work in Hawaii on Columbia/TriStar's upcoming CG sci-fi feature, Final Fantasy: The Spirits Within, based on a popular video game. That film has a specific goal of getting as close to photo-real as possible, according to Andy Jones, the project's animation director at production company Square USA, Honolulu.

“We know you can't achieve perfect photo-realism, but we wanted to pursue a new style of CG close to photo-real, but flavored like [2D Japanese] Anime,” says Jones. “The company has been in R&D to achieve this goal since 1997. In that time, we have built specific facial-animation tools to make our characters more lifelike. We've developed a new skin shader, muscle movement under skin, moistness in the eyes, peach-fuzz hair on top of the skin, and so on.” (For more on Final Fantasy see next month's Millimeter.)

Such breakthroughs are hardly limited to feature films. Many studios are applying development work initiated for larger projects to their small-screen work. ILM, for instance, recently refined its metallic texturing tool used on features to animate a Pontiac commercial in which a giant, metallic cat chases a Pontiac.

“The client wanted the cat's movements to be fluid, and so we based them on reference footage of cheetahs, lions, and tigers,” says Tim Stevenson, ILM's CG supervisor. “His exterior, though, had to have a metallic sheen. We decided to use scanned metals as a base to texture the cat, but on top of that, we used a brushed metallic texture that we had developed. It gave the cat a stainless steel look, with tiny scratches along the surface. We developed a special shader to create that look, emphasizing specular highlights, but not too shiny.”

Those animating characters outside the umbrellas of monolithic studios, however, usually don't have the cash or infrastructure to develop the proprietary tools readily available to ILM, Disney, DreamWorks, and a few others. Many, of course, rely heavily on innovations contained in the main, off-the-shelf, animation packages. Indeed, tools like Maya, Softimage, 3D Studio MAX, and LightWave have essentially revolutionized character animation in recent years, particularly since the arrival of inverse kinematics to connect movement to a character's skeletal structure.

But many character animators also credit “smaller tools” — third-party plug-ins — for making CG characters more widespread.

Animator Chris Zapara, for instance, credits a LightWave plug-in called Relativity — developed by engineer Prem Subrahmanyan — with being crucial to automating the gigantic sandworm featured in the Sci-Fi Channel TV movie, Dune. Zapara, then working with Area 51, Burbank (he is now at Foundation Imaging, Valencia, California), had few time or budget luxuries while developing unique movement cycles for the worm.

“Much of the worm's movement happened because of Relativity,” says Zapara. “It's a tool that lets you automate complicated moves. Without it, we would have had to manually begin every shot from scratch, which would have been tough on our deadline. When you have 80 to 90 shots to do, you need tools to take your animation work and replicate it quickly.”

“Producers were afraid the worm would end up moving like a snake, so we took out a lot of initial side-to-side movement,” Zapara continues. “That helped because he was too large to wriggle side-to-side too much. On the other hand, too little movement left him static, like a log. So we finessed it, moving motion in and out until we had unique movement. With Relativity, once we came up with the moves we all liked, it was easy to automate the process so that when the worm's head moved, his body followed.”

John Teska, visual effects director at Foundation Imaging, has had similar experiences. He insists that the animation community's drive to address specific end-user problems with such plug-ins has been crucial to the recent proliferation of CG character animation.

In both the quests for photo-real and photo-fantastic characters, the ability to reach into the real world and lift textures for insertion onto animated characters has proven priceless.

“We ran into lots of skin-texture issues on Final Fantasy,” says Jones. “We developed [proprietary skin shader] SQFlesh to plug into RenderMan specifically to address the issue of how light interacts with skin from various angles. Light behaves uniquely on human skin, so animating the way it enters the skin, bounces around just under the surface, and provides a subtle warmth to the skin, along with subtle shadows, was important. Our shader automates a lot of that process.”

Bob Forward, who has produced several animated television shows, is currently working with Foundation Imaging on an upcoming, syndicated, CG space show called Dan Dare. Forward says the art of texturing has “greatly improved character work” in recent years.

“Software, in general, has gotten better, but the whole process of texturing is far more sophisticated now,” Forward says. “Our animators now routinely take digital photographs of actual human skin, and in the computer, they peel that off and apply it to characters. That's why our characters will have real human skin, pores, freckles, and so on.”

The choice of real-world textures for animated characters, though, may not always line up with a real-world counterpart. Zapara, for instance, also faced dilemmas of how to texture Dune's giant worm. In the end, he settled on wood.

“The original worm model was sort of brown, so we tried bark textures, and ended up with bark as our primary source,” Zapara recalls. “I modified it a bit, creating more fleshy undersides, where the worm's scales overlap.”

For humanoid characters, the debate over motion capture rages on. The industry largely seems to be splitting along TV-film lines — heavy use of motion capture for TV animation, general reliance on keyframe for feature films.

Indeed, most completely CG television shows have, by now, turned to mo-cap. Even Mainframe Entertainment, Vancouver — the company that pioneered production of all-CG children's programming using meticulous keyframe methods — has joined the mo-cap world. Last year, Mainframe purchased a Motion Analysis, real-time, optical system and promptly moved into production on two, new series, Action Man and Heavy Gear, as well as an upcoming Reboot TV movie.

“We still primarily depend on keyframe animation for facial work,” says Alastair Macleod, senior motion capture technical director at Mainframe. “But for body movement, we largely use mo-cap. The reason we did it, though, is not so much related to cost or speed. We made the move because that is what clients were requesting. Hasbro [producers of Action Man] felt motion-capture was established enough to give them the look they wanted, so they asked us to produce the show that way. Once we bought the rig, it made sense to use it for other projects, as well.”

Macleod says that motion capture has permitted Mainframe animators to move past mundane functions to more complicated work.

“It makes sense for us to mo-cap things like basic walk cycles,” he notes. “Even with a show like Heavy Gear, we have found that we can create basic movement cycles this way, even though the characters are not human — they are giant robots. We wanted more robotic movement, obviously, but we found that by putting prosthetics on the actors on the mo-cap stage — ski boots and hockey gear — we can get robotic movement with the mo-cap system and save some time. But particularly for walking, running, and standing up, motion capture has proved useful.”

Foundation Imaging, which relies on Vicon 8 optical technology, uses mo-cap mainly for children's shows. Teska suggests that the concept of building motion libraries will likely evolve into a logical next step for facilities like Foundation Imaging, which possess both sophisticated mo-cap stages and huge CG character workloads.

“We've had such a huge body of motion come through our shop, it seems obvious we could use some of that data as challenges come along,” says Teska. “If you know where to look, you could probably build a huge library and use it regularly on new shows. Eventually, it would make sense to catalogue that stuff more than we do now. We have already used generic walk cycles for different projects, Starship Troopers: Leathernecks and Voyager in the same season, for instance.”

Commercials are less apt to take advantage of motion capture because of short turnaround times and the need to stylize spots. Sometimes, though, mo-cap can bring unique performances to even simple, 2D animation. Late last year, Mary Kay Cosmetics hired Computer Animation Technology (CAT), Dallas, to capture dance moves from three dancers with its Vicon 8 system, not for insertion into detailed computer models, but for use with 2D brush-stroked illustrations.

“The characters were paintings, and the commercial was more of a painting with moving images,” says Trent DiGiulio, president of CAT. “They wanted these little brushed stick images choreographed so they would interact in a way that would be difficult to replicate with keyframe animation, to give them human motion. This was the furthest thing possible from photo-real animation. The end result is very stylistic animation, not what you would normally associate with motion capture.”

Feature filmmakers, however, usually have years to produce projects, and so, studios tend to rely on keyframe techniques. While New Line Cinema is currently permitting vendor Weta Digital of New Zealand to use mo-cap for its upcoming Lord of the Rings trilogy, and a handful of low-budget, indie films are using mo-cap tools, most big studio pictures still keyframe characters.

“For films [like Shrek], there is usually discussion and tests relating to motion capture at the start, and that is what we did. But in the end, we chose to keyframe everything,” says Adamson. “I think it's a stylistic choice. For a movie like ours, and many others, you want movement to be stylized, different than real-world motion. Our lead character, Shrek, for instance, is a big green ogre, and we wanted him to move something like a lumbering, fat guy, but with a few differences. In that case, keyframing made more sense.”

Final Fantasy, on the other hand, relies on both motion capture and keyframing for its characters. The decision about which approach to take for which shots depends, Jones says, on the movement in question.

“We mainly use motion capture for larger, quicker actions, a lot of action sequences with running, fast walking, jumping, and so on,” explains Jones. “For subtle movements, close-ups, facial work, basic joint movements in the hands or feet, we keyframe. The goal is to make seamless transitions between those shots.”

Although facial motion capture has made strides in recent years, most character-heavy projects still do the bulk of their facial work with a keyframe approach.

“[PDI] has taken a different facial approach with Shrek than we did with Antz,” says Hui. “In Antz, we did not have all major facial muscles built in, under the skin. But for Shrek, we have more advanced proprietary tools for this purpose, so the characters have more muscles working in their faces and even down their necks. When Shrek moves his head, you see the fat rotating and bulging on his neck. We spent a long time writing code to improve this aspect. We also wrote a new lip-synch software, with a facial library that lets animators lock mouth moves to the soundtrack and then simply nuance those moves as needed. Our most photo-real human character, Princess Fiona, has more muscles around her eyes and mouth than anything we've ever done before.”

But the biggest breakthrough on Shrek may be its most subtle. “Fiona's mouth features that characteristic that happens when your lips stick together and then come apart for a second when you open your mouth,” says Adamson. “We feel we made her speech and mouth convincing by adding this characteristic. We did it with a proprietary tool we call ‘Lipstick,’ which controls the opening of the mouth, the compression of the lips, and how much and for how long they adhere to each other.”

Final Fantasy's Jones is also proud of that film's progress in the area of the human eye. He says such advances resulted from years of observation combined with proprietary tools.

“The big challenge with eyes, we found, was in timing — when people move their eyes in the middle of a sentence, how often they blink, and so on,” says Jones. “In particular, animators often make a common mistake of thinking that when a head scans across a room, the eyes move in concert with the head. Actually, the eyes make a lot of subtle ‘hits’ as the head is moving, darting in different directions. We built a proprietary system that drives that movement, moistens the eyes, and moves the skin around the eyes as the eyeball itself moves.”

All these issues, of course, are central to the ongoing industry debate over so-called “photo-realism,” and whether it's even a worthwhile goal.

Certainly, recent experiments — such as the LifeFX motion-capture breakthrough a couple of years ago to create an impressive, completely digital face of actor Jim Carrey for the currently shelved feature film, The Incredible Mr. Limpett — have proven how tantalizing such a goal is.

On the other hand, leaving aside questions about whether such complicated and time-consuming approaches are cost-effective in modern production settings, many industry professionals consider photo-realism a moot search. Even Jones admits that, despite publicity suggesting that Final Fantasy features “photo-real” characters, they are really “as close as possible to photo-real, but it's still obvious this is an animated movie.”

According to Pixar's Catmull, photo-realism is “simply not relevant.”

“Anything animated, you know it's an artistic creation, and therefore not reality in these films,” says Catmull. “Therefore, from an animation point of view, why bother worrying about photo-realism? It's not important. At Pixar, we'll make whatever image we need to make to tell the story, but we aren't about to pretend they aren't animated.”