Step By Step: I, Robot

In bringing I, Robot to the screen for 20th Century Fox, director Alex Proyas held some aces — the appeal of actor Will Smith and the visual effects supervision of Oscar-winner John Nelson. But one essential element was unknown: Whether the performance by a main character called Sonny could hit the necessary emotional chords. Sonny had to hold his own with other actors, despite the fact that he was a semi-translucent robot fashioned purely of 3D-CGI at the Venice, Calif.-based Digital Domain.

“Like an Apple iMac computer” is how Nelson describes the robot's look. “There was no way our robots could be guys in suits.” Because Sonny had to emote, Nelson says, they approached him the way Peter Jackson approached Gollum on The Lord of the Rings. Proyas used actor Alan Tudyk as a proxy for Sonny on-set, and footage of Tudyk's performance, coupled with motion capture of his body, became the foundations for DD's animation.

One scene in which Sonny speaks to actress Bridget Moynahan had to convey a key story point — the robot's realization that ”I am unique.” For the scene to work, Nelson says, “You have to see all the nuances of his expressions.” There was no facial motion capture, however, since the robot's face was so different from the actor's. “We had two 24p DV cameras on either side of Alan at 45-degree angles to capture his performance in close-up,” he says.

“We positioned the video cameras to get as clear a shot as we could from different angles so that our animators would have different views — in synch — of Alan's performance,” says Erik Nash, DD visual effects supervisor. “If the film camera didn't provide an ideal angle, hopefully one of the video cameras would have a better angle. Having different views helped the animators see what was going on in three dimensions.”

“It was the best way to capture the subtleties of Alan's facial performance,” says Andy Jones, DD animation director. “It also came in handy to reference what he was doing physically — like how many steps he took — when we had to recreate those movements in motion capture. We'd review the cut and then Alan would mimic that as much as possible.”

The process of having Sonny interact with real actors required multiple steps on-set. “We would shoot Alan with the other actors, and then pull him out and shoot the other actors with no proxy for the robot,” Nelson explains. “We'd also shoot with a 6ft.-tall robot stand-in to see how the plastic material would look in the lighting.”

According to Nash, DP Simon Duggan lit the scenes for the actors. “But what made the actors look good invariably made the robot look lousy,” Nash says. “When we'd roll out the stand-in robot to get a quick burst of film, Simon would lean over to me and quietly say, ‘You're gonna make that look better, right?’ One of our technical challenges was making the robot look believable in actual environments.”

To capture lighting information about a scene into which the CG robot would be inserted, Nelson's team shot reflective chrome and gray balls and HDRIs (high dynamic range images), which DD would later use for image-based lighting of the character. They came up with a new tool called Robo-Tile. “[Robo-Tile] was for shooting fisheye environment maps in high dynamic range using a digital camera. It had a little portable motion control pan/tilt head. We would shoot four angles with a fisheye lens that we would then stitch together into a complete spherical map of the environment. We'd shoot multiple exposures in each one of those four positions, and when those images were stitched together that gave us all the luminosity information in a particular location — all the way from the deepest shadows to the brightest lights.”

Those spherical maps were useful for reflecting the environment onto the robot's slick surfaces, too. “We had 360-degree views of our environments. Without that, we would have had to use old techniques like reflecting plate photography, which never works well,” Jones says.

Animation was done in Alias' Maya, with much of the facial expressions handled frame-by-frame. “Because the robot's face is so completely different than Alan's we couldn't just mimic the actor's movements to deliver the same feeling,” Jones says. DD also had a scan of the plastic robot stand-in, but when it came to modeling nuances like the robot's brow movements, he says, “we were on our own.”

While Sonny's face moved, the inner workings of his head also had to be visible, but not interfere with his facial expressions. “The challenge was maintaining the semi-translucent quality yet having the facial surface visible enough that you could read the subtle changes in shape that give you a performance — the arch of an eyebrow, the flaring of nostrils, the curve at the corner of his mouth,” says Nash. “It was a real tightrope walk to make it look photoreal.”

Temp animation was rendered in Maya, with finals done in Pixar's RenderMan.

Clever use of shaders created the subsurface scattering of light beneath the robot's translucent surfaces. But with 17 layers required for each robot, DD used a custom plug-in for its Academy-honored Nuke compositor that allowed great flexibility in how those rendered elements could be assembled. Called Make-Bot, it enabled DD's compositors to use sliders to control various parameters. “Other than primary lighting direction and ratios, all other aspects of the robot's look — its shine, its transparency — were controllable in compositing. We didn't have to go back and re-render for every little change,” Nash explains.

The end result, says Nelson, was a dynamic character that reflects the actor's art, the director's eye, and skillful animation and digital effects. “In so many ways, what we've made is a robotic movie star.”

Director - Alex Proyas DP - Simon Duggan Visual Effects Supervisor - John Nelson Digital Effects Supervisor - John Berton Robot Designer - Patrick Tatopoulos Motion Capture - Motion Analysis For Digital Domain: Visual Effects Supervisor - Erik Nash Animation Director - Andrew R. Jones Digital Effects Supervisor Computer Graphics - Jonathan Egstad Supervisors - Serge Sretschinsky, Vernon R. Wilbert Jr. Compositing Supervisor - Sonja Burchard Lead Animator - Matthew T. Hackett Character Set-Up Lead - Steve Preeg Technical Director - John Hewitt CG Robot Lighting Lead - Paul George Palop