Integrated Team Effort

 Web Expanded

Shot Breakdown from The Golden Compass

Understanding lighting schemes was key to making the robots of Transformers look realistic. Photo: Industrial Light & Magic © 2007 Dreamworks LLC. and Paramount Pictures. All Rights Reserved.

“Collegial” isn't usually the first word that springs to mind when describing the competition for Academy Awards, but in this year's visual-effects category, that word fits well. The three lead nominees — Michael Fink (The Golden Compass), Scott Farrar (Transformers), and John Knoll (Pirates of the Caribbean: At World's End) — have been industry colleagues for decades. Farrar and Knoll have worked at Industrial Light & Magic (ILM) since the 1980s, and both have collaborated on films with Fink, an independent supervisor. While their latest projects couldn't be more different in style, these pros share an approach to visual effects that reveals a lot about the state of the art.

Most notably, all three supervisors come from photographic backgrounds, and their knowledge of cameras and lighting is reflected in each of the nominated films. “The technology is now here to do incredibly accurate lighting,” says Fink, a previous Oscar nominee for Batman Returns and this year's Achievement in Visual Effects winner for The Golden Compass. “That's where the art is.”

Integrating effects within a film's photographic style is their key concern. Knoll, whose work on the Pirates franchise has earned him three Oscar nominations and last year's statuette, says he believes that lighting is crucial to making effects blend with the look created by a DP. “We pick up where the DP leaves off,” says Knoll, who began his ILM career as a cameraman. Although he's computer savvy (he co-authored the original Adobe Photoshop), Knoll sees advantages in having a live-action background. “Experiences on set, like holding a light meter and figuring out how to do an exposure split, teach you where the tradeoffs are. It helps you avoid mistakes like having interiors at the same brightness as exteriors,” he says. Supervisors having only CG backgrounds can be vulnerable to such errors, Knoll says. “For example, the standard cameras in animation systems rotate around their nodal centers. And that almost never happens in the real world.”

Transformers provides a striking example of live-action approaches to illuminating visual effects. Farrar, an Oscar winner for Cocoon, approached Transformers by imagining how a DP would have photographed 20ft.-tall robots if they really could stride through a shot. “If they were really on set, they'd be lit separately when they're in closer view,” he says. “The DP would knock out the sunlight for the close-ups and have HMIs [Hydrargyrum medium-arc iodide lamps] and cutters to cause shadows. You wouldn't just be worried about rendering the background lighting. Of course you need to reflect the background, and we recorded the robots' environments with reflective spheres — that's a given. But we totally relit the robots to look great wherever they were.” To accomplish this, ILM created virtual versions of cutters, flags, bounce cards, and shiny boards. “The robots could move in and out of shadows, which generally isn't done much in computer-graphics lighting,” Farrar says.

The visual-effects shots of The Golden Compass were tracked and shared by nine post facilities.

This year's nominees also demonstrated that locked-off visual-effects shots are increasingly rare. “We try to conform stylistically to the rest of the movie,” Knoll says. “Pirates was never shot from static cameras, so we didn't want the visual effects to take on a different look. You don't want to feel like you've entered a part of the movie that was heavily storyboarded. … The state of the art has advanced to the point where you can kind of not worry about the camera. Tracking software has become more sophisticated, and crews have developed very good methodologies for solving complex camera motion. We don't think twice about Steadicam or handheld shots. That makes for better shots, because it's impressive when you see CG happening in the background of a Steadicam shot.”

That is not to say that current tracking methods have become standardized, however. For The Golden Compass, shots were shared by several facilities — including Framestore CFC, Rhythm & Hues, Cinesite, Digital Domain, Tippett Studio, and Rainmaker. “Everyone does it differently,” Fink says. “When a background would go to another facility, it would be re-tracked. There's very little sharing of motion data. Of course, these facilities were usually tracking different parts of the frame.”

In Transformers, where the camera moved rapidly through cityscapes, Farrar employed photogrammetry to create backgrounds racing by. “It's a photoreal movie, so we photographed as many real buildings as possible. When we used computer models, the textural information was actual photography. We'd hang a camera over a street and photograph it in all directions. There were shots where a camera was flying down the street, and the shot was assembled from still photos. We'd relight it and add interesting shadows and maybe even change the colors of buildings to make it more appealing,” Farrar says.

Facial animation of CG-enhanced character Davy Jones was done by hand after carefully studying what actor Bill Nighy was doing in Pirates of the Caribbean: At World''s End. Photo: © Disney Photo Enterprises, Inc. All Rights Reserved.

The characters in this year's nominated films couldn't be more different, but the supervisors' animation strategies shared some commonalities. The CG robot characters in Tranformers, sporting layers of glossy paint and thousands of moving parts, were accomplished with brute-force animation. “We had 48 different transformations, and every one was a one-off,” Farrar says. “There was no way to do it procedurally.”

Pirates also relied on significant hand animation, especially for the dazzling CG speaking character Davy Jones. While ILM employed some motion capture of actor Bill Nighy, Knoll says, “We didn't even attempt facial animation. It was done by hand after carefully studying what Bill was doing.” Knoll says he is skeptical that facial capture would have worked anyway. “The capture process itself is very lossy. If you compare images of an actor during capture and the animated results, it's as if you're looking at an ‘emotional blur filter'' — like the corners of the performance have been rounded off and you only have about 70 percent left. That's what looks creepy.”

Photoreal animals were the central challenge of The Golden Compass, which features dozens of scenes with CG bears, monkeys, wolves, ferrets, raccoons, and dogs. One leading bear character had 7 million hairs. “Fur was still a challenge,” Fink says, adding that Framestore CFC came up with new approaches to creating muscle and fat and skin to make the bears move convincingly. “What was groundbreaking was the level of interactivity between the CG animals and the live actors. There would have been no story without those relationships.”

While simulation strategies were employed judiciously to jiggle CG springs in Transformers and ripple animal pelts in The Golden Compass, simulation was a dominant technique in Pirates. In particular, a half-mile-wide maelstrom at sea has arguably advanced the art of CG simulation considerably.

“Previously we've done different types of simulations — cloth, particle, rigid-body, fluid — by using separate engines optimized for specific purposes. But it's becoming more and more necessary that things interact with each other,” Knoll says. “So a few years back, we switched over to the Stanford University-based simulation engine, which supports all of these different types of simulation. So we no longer have to do all sorts of tricks to fake that interaction.” (For an in-depth discussion of fluid-simulation strategies, see p. 30.)

Despite all the digital developments on display in this year's Oscar-nominated films, there were still essential contributions made by traditional set-effects experts. It's a measure of their contributions that Trevor Wood was named among the nominees for The Golden Compass,while John Frazier earned Oscar nominations for both Pirates and Transformers. Fink says that Wood built a motion base that enabled a child actress to ride on the back of an animated bear. “Trevor built all the actuators and mechanical bits in CG first and then we watched the bear animation with this machine animation inside, to make sure that what we were building would do what we needed,” Fink says. “He then took the CG data, exported that to a computer-controlled machine, and machined the parts to make the motion actuators. And it worked right out of the box.”

Frazier also faced a major mechanical challenge in Pirates, building a huge gimbal that could roll 110 degrees. Knoll says he felt that ILM could use physical ship models because digital tools could correct traditional drawbacks. “Today's technology can fix problems like depth of field and water droplet sizes. We got to have the fun of shooting big boats in a tank, and you couldn't tell they were models. It was the best of both,” Knoll says.

On Transformers, countless cars had to be flipped as though they were pushed by rampaging robots, while miniature buildings were demolished by robots run amok. But Farrar says that shooting elements separately and compositing them in later is still not trivial. “You have to distort them like crazy if they're going track correctly in the shot,” he says.

An undeniable reality of today's visual-effects-heavy films is that they often require collaboration among multiple facilities. Knoll employed Digital Domain and Asylum on Pirates, using teleconferences to foster communication. But The Golden Compass may have set records, with nine shops participating. Fink calls the process “parallel post,” and he says that some shots include the work of four different facilities. “We had a team of coordinators who did nothing but shuffle elements,” he says. The team used cineSync conferences and Sohonet hubs in Los Angles and London to transfer data. “We had so many 2K files flying around that FTP sites were not adequate. FedEx saved us in the end.”

So the degree of difficulty in modern visual effects is measurable on several fronts. That's unlikely to diminish, but Farrar thinks the best visual effects can be evaluated with one simple question: “Is it a good idea?” Quoting multi-visual-effects-Oscar-winner Dennis Muren, Farrar jokingly says, “‘We can do anything.'' We just don't want to!”

• Michael Fink shared this year's Academy Award for Achievement in Visual Effects for The Golden Compass with Bill Westenhofer (Rhythm & Hues), Ben Morris (Framestore CFC), and Trevor Wood (special effects).
• Scott Farrar shared the nomination for Transformers with Scott Benza (ILM), Russell Earl (ILM), and John Frazier (special effects).
• John Knoll shared the nomination for Pirates of the Caribbean: At World's End with Hal Hickel (ILM), Charles Gibson (visual effects supervisor), and John Frazier (special effects).