Dr. Manhattan Project

To create the Dr. Manhattan character for Watchmen, Director Zack Snyder and Visual Effects Supervisor John “D.J.” Des Jardin opted to build actor Billy Crudup a motion-capture suit that incorporated a densely packed mesh of blue LEDs in order to provide an interactive light source on set consistent with the character''s nature. They then used Crudup''s live-action performance as reference for a largely hand-crafted CG character, which was built at Sony Pictures Imageworks.
All photos TM & © DC Comics, courtesy of Warner Bros. Pictures.

Whenever embarking upon the creation of a “spectacular smorgasbord of visual craziness” in a major feature film — as Director Zack Snyder describes the visual effects in his new movie, Watchmen — it's usually a good idea to make sure you can actually accomplish what you envision. This was the prickly question facing Snyder and his collaborators when they first began developing a plan for making the superhero-themed, graphic-novel-inspired movie.

The film, slated at press time to debut in March, is based on a legendary comic series created by Alan Moore and drawn by Dave Gibbons (who served as a consultant on the project) that portrayed the adventures of a group of costumed vigilantes in an alternate version of the United States in the 1980s as the nation hurtles perilously close to nuclear war with the Soviet Union. To pay homage to Gibbons' iconic imagery, Watchmen features almost 1,000 shots involving digital effects, which were crafted at four visual-effects facilities (Culver City, Calif.'s Sony Pictures Imageworks; The Moving Picture Company's Vancouver, B.C., facility; Toronto's Intelligent Creatures; and CIS Hollywood). A host of images — the destruction of cityscapes, a glass palace on Mars, one character's unique flying ship known as the Owl Ship, the organic mask of another character called Rorschach, and much more — all posed unique challenges for the visual-effects team, under direction from Visual Effects Supervisor John “D.J.” Des Jardin.

The vexing question as the project got under way, which had to be solved ahead of all other challenges, was how to realistically and logistically create the iconic and extremely powerful character known as Dr. Manhattan. In both the graphic novel and the movie, Manhattan is a physicist whose body is ripped apart until he reconstructs it as, essentially, a perfect male specimen — teeming with unlimited energy, constantly glowing blue, and able to grow to unlimited heights. Picking a methodology for creating Dr. Manhattan and confirming through tests that it would be both feasible and believable on the big screen was, in Snyder's words, the central issue “that made the entire movie viable.”

Cinematographer Larry Fong (pictured, center, with Director Zack Snyder, right) used two Sony HDW-F900 HD cameras—one targeting body motion and one targeting facial motion—in phase sync with Fong''s 35mm Panavision cameras to capture detailed reference information for animators.
Photo by Clay Enos.

That testing took months and included several ideas that were considered and then discarded along the way. “We were putting a photoreal, 30ft., blue, naked man glowing with energy — who is, himself, a light source — on the big screen,” says Cinematographer Larry Fong.

Indeed, allowing Manhattan to be an interactive light source himself was central to the project's credibility, filmmakers insist. The nature of the original character in the graphic novel caused them to eventually decide he would have to be an all-CG creation — even though his performance largely revolves around conversation, close-ups, and interrelating with other characters, rather than action or stunts.

“Zack had ideas presented to him that involved painting an actor with a blue material and doing a 3D process afterward, but I felt that did not help with the problem of him being over 40ft. tall and then shrinking down to be a normal guy all in a conversation, or him growing giant to fight Vietnamese soldiers, or creating two or three versions of himself in his apartment,” Des Jardin says. “We would need motion control, among other premeditated and time-consuming production tricks. So, quickly, as we talked about it, Zack asked about him being an all-CG character, and I said, ‘It solves a lot of our problems with shooting him and putting those shots together.'' With a CG character, you don't have to worry about multiple [motion-control] passes in the sequence where there are four of him or all those strange techniques to make him so tall.”

Still, CG or not, Snyder insisted Manhattan be played by a real-life, A-list actor: Billy Crudup. Thus, filmmakers initially began gravitating toward a motion-capture concept similar to what Weta Digital used for the Gollum character in the Lord of the Rings films, what Industrial Light & Magic (ILM) used for Davy Jones in the Pirates of the Caribbean films, and what Imageworks itself used for The Polar Express and Beowulf. However, further tests soon helped them conclude that motion-capture data and high-resolution HD video of every aspect of Crudup's performance could be best used not to drive the performance, but rather to reference and inspire it — so Imageworks' animators animated the performance by hand instead. But because they also needed Dr. Manhattan to be an interactive light source, they eventually moved toward the idea of using the tools and some of the processes of the motion-capture world to build Crudup a special suit that would allow him to become a human lighting instrument, interacting with the characters and sets around him at the exact same time they were recording his performance for reference purposes on set.

The HD viewing system on set allowed Snyder (pictured with Fong) to immediately
see, absorb, and discuss Fong''s compositions, lighting approach, and color considerations
throughout production, which was particularly helpful where the capture of shots involving
Dr. Manhattan were concerned. Photo by Clay Enos.

“In other words, we wanted the actor on set, casting a blue light and interacting with the rest of the scene. And then, we would replace him with [the CG] Dr. Manhattan,” says Pete Travers, Imageworks' supervisor on the project. “That meant we had to build [Crudup] a suit of light that would emit off him and cast light into the scene. We even used it as an off-stage light source and to light close-ups of others in dialogue with Dr. Manhattan. We called this above-the-line lighting since we were getting it from our lead actor.”

Travers says the production team examined all sorts of concepts — reflective paper that could be worn, soft stage lighting, and other things. But in the end, only a densely packed mesh of blue LEDs seemed to create the on-stage effect they were searching for.

“I had been experimenting with LED lights and told Zack I thought they might be a good source, without a lot of heat associated with them, to put on a performance-capture kind of a suit,” Des Jardin says. “We decided that we could litter a suit with LEDs, even thousands of them, and that could be the source of the glow so that [Dr. Manhattan] could reach out and touch someone, and it would seem like a diffuse glow of light, not separate light sources. This way, also, light from his chest could fill in on his face. The diffuse nature of the light also allowed us to keep his face free of LEDs for facial-capture purposes, since the light from his chest and head cap provided fill for his face. We tested this idea from about May to mid-July [2007] in a prototype suit that Chris Gillman of [practical effects firm] Global Effects built for us, and we decided it would work. Chris and his team then built four other hero suits that consisted of a head cap, body suit, gloves, and shoes with LEDs on them, so we could even see light emanate from the soles of his feet when he walks.”

The suit created by Global Effects achieved three important goals: It allowed filmmakers to get the right light throw to use as real interactive light in scenes, it was flexible enough to allow Crudup to act without feeling encumbered, and it was trackable. In addition to the LED configurations, filmmakers put unique pattern-tracking markers on it so that HD footage could easily be used as reference to perfectly capture Crudup's performance.

This approach allowed filmmakers to record Crudup's every move using two Sony HDW-F900 HD witness cameras — one targeting body motion, one targeting facial motion — in phase sync with Fong's 35mm Panavision cameras (he shot the entire film using Kodak Vision2 500T 5218 35mm stock and Vision2 100T 5212 stock for day exteriors), with the HD cameras set to high shutter speeds to minimize motion blur for tracking purposes. As Fong filmed the scenes, he captured all sorts of interesting reflections and bounces from the blue LED glow emanating off Crudup's body — much more detailed lighting than the team had been anticipating.

“[The lighting yielded] tons of stuff for free,” Snyder says. “Some of the reflections we got — light reflecting all the way into a pitcher of water, amplifying it at some distance away — those are things we wouldn't have done in post because our resources would have been somewhere else. We got all of that for free.”

The glow from Crudup''s LED suit yielded much more interesting and detailed lighting than the filmmakers had expected. According to Snyder, the team would not have had the resources to replicate the reflections on surfaces surrounding the character in post.
Middle photo by Clay Enos

“It took some R&D because we had to go back and forth to figure out the density and distribution of the LEDs, since they are really pinpoints of light,” Travers says. “They had to be distributed somewhat evenly on the suit in order to get us an even throw of light. The unevenness of the light actually worked in our favor for interesting light play, especially in the love scene. But there are a number of times when [the character] has interesting, close contact with someone and the light plays on their face a certain way. With one character [Silk Spectre], her face is a certain indigo skin tone and when his glove gets closer, her face goes completely cyan. It was surprising, some of the things that light did. What is interesting about LEDs is that sometimes the light color is indigo, but the closer you get, they turn cyan. In fact, when we did do artificial environment lighting in some places, we tried to match that kind of feel — that idea of having a hue change based on the distance of the light source. That made it much more realistic. The end result was that we got plate photography with character source lighting that was very precise, and almost impossible to replicate as a post process.”

“[As a result of all this,] I never had to do any special lighting when Billy's character was in the room,” Fong says. “We just did it like any character, depending on what was appropriate for the scene. If it was a bright situation, such as in daylight, the glow should be less noticeable, which happened naturally. Other times, like during the love scene in bed, the character himself was pretty much the primary light source. The only time we needed more than that was when we were filming the bigger set pieces — scenes where Dr. Manhattan is gigantic in his lab. When he was gigantic, we had to find ways to pump more light to match his increased size. We added more bounce light in those situations, and for some of them, we had a huge HMI balloon at the top of the room to represent the light he was supposed to be emitting at that height.”

While achieving the production's lighting goals and allowing it to shoot on location without encumbering itself with the technically heavy footprint of a performance-capture stage, this approach still left filmmakers with the painstaking problem of painting out the real Crudup and replacing him with his CG doppelganger. This process was largely accomplished traditionally using rotomation and paint techniques at Imageworks, while the animation team created Dr. Manhattan's performance based on reams of reference material.

“Normally, you paint a good, clean frame of something, and you track it typically, but that can get thrown out the window when the character provides light and is moving around,” Travers says. “Light can change subtly every time in those shots, so some of our typical solutions for roto and paint didn't always work. There were also size differences, because Billy is about 5' 8", while the character, at his normal height, is about 6' 2" and very muscle-bound. That said, we only had to paint out where [the CG] Dr. Manhattan would not cover Billy. So the real work was in painting Billy from the scene. But it paid off, because we have a CG body and face with a real actor's performance and a color palette that came out of the real on-set photography, including interactive lighting.”

At this point, Snyder, Des Jardin, and Travers all come back to the method of creating reference for the character using Crudup's performance as the reason they could make what they consider to be a believable digital human — albeit an unusual one — out of Dr. Manhattan.

“We've been on the digital human project for years,” Travers says. “We've made real good advances in rendering technologies and look development. But where I think most CG humans fail is in the animation. So we made a point to make that a clear and heavy aspect of this show — to give the animation team a guide to go by. In this case, we filmed an A-list actor at close range, which provided excellent performance reference. After all, you would never ask a master oil painter to paint a masterpiece with no subject reference. Our pipeline for judging the animation was done splitscreen, CG doc side-by-side with Billy. The idea was to match as close as we could, but with a principal goal that we would not be slave to any particular technology, so if we wanted some motion-capture data for a particular thing, we used it, or if it was best to just animate by hand, we chose that course.”

About 10 percent of Dr. Manhattan is mo-cap, and the bulk is hand-animated. The character was animated in Autodesk Maya, with Imageworks using Apple Shake and Side Effects Software Houdini for compositing, along with several proprietary tools.

The effects team kept Crudup''s face free of LEDs and instead spotted it with mo-cap tracking markers to capture nuanced reference data from which to build the character''s facial performance.
Photo by Clay Enos.

More specifically, parts of Manhattan's facial movement are mo-capped. Since the LEDs were limited to Crudup's body, reflecting a glow on his face, filmmakers were free to spot the actor's face with mo-cap tracking markers — about 200 of them all over his face, using some of the same concepts used by Imageworks for Beowulf.

“We took that into Maya and it was a primer coat, so to speak, and then we did some fine-tuning of the performance,” Travers says. “But most of the performance is from skillful animation designed to match the reference and lots of rotomation — 3D tracking of the character's performance, matching what we had on set, using a kinematic skeleton, basically using the character rig. Since [Dr. Manhattan] is naked and Billy was covered in a suit, we did all sorts of extra reference shoots on how muscles moved. Usually, we would get the body performance all tracked in, and once we were close, we would work on the facial stuff and get into all kinds of secondary dynamics like jiggle and high-end deformations.”

The facial work was so detailed not only because of the goal of making the character as photoreal as possible, but also because he is unique for a CG character in that he largely engages in intense conversations in the movie, rather than in significant action. Thus, facial close-ups of the character are routine, and important, in Watchmen.

“CG characters usually jump around, climb buildings, break things, and go nuts,” Snyder says. “But Dr. Manhattan, much of the time, the biggest thing he does is furrow his brow. It's Billy's acting that we needed — subtle stuff. So, for me, the thing I was most worried about was getting those emotions to play on his face.”

Des Jardin says his team worked with Imageworks to do some early facial tests on the project using computer models from Beowulf, and they decided to mix some mo-cap data with hand animation largely because it was the most practical approach for the project's timeline and resources.

“I can't say our technique was better than entirely performance-capturing the face, but traditional mo-cap doesn't lend itself to how we were making this film,” Des Jardin says. “That array of cameras, in a set environment, wouldn't work on location all over Vancouver, and we had no time or ability for a crew to rig each location with a mo-cap array. So the approach we used is a lot more hand-intensive. But either way, the thing that drives the rigging of a character's face is: How many subtle things can you get out of it? Is it 100-percent real? The way we did it was to essentially copy a real performance. So, like with motion capture, Billy was totally driving the performance as long as we did our jobs right, and Imageworks did that. They did an amazing job with his skin, thin layers of peach-fuzz hair, and other things. He's a blue, glowing guy, but he's also supposed to be a real guy, and I think you believe he is when you see him.”

Among other things, central to making Dr. Manhattan's face move realistically was the time the animation squad put into analyzing and emulating the movement of Crudup's neck. The goal was to capture the realistic movement of tendons rolling; his Adam's apple moving; and his skin sliding across the muscles, tendons, and bones beneath.

“One thing about our approach: We realized that neck movement is critical to facial movement,” Travers says. “Often, facial animation is too localized to the obvious places, like the mouth. We analyzed movement of the tendons and other structures within the neck area, and that led to making the facial performance believable. We spent a lot of time getting our rig to move like Billy's neck, and that was why we kept his neck clear of LEDs and used it as a major reference. I think it paid off. The system slides the skin as the Adam's apple moves, and if he tenses his neck, it slides over the clavicle a certain way. The rig was somewhat cumbersome at first, but we had layers of animation, so we could do the basic body layers first, and then the neck rig is sort of built inside-out and can be applied to slide that skin around. But I want to emphasize: This wasn't based on theoretical discussions. It was based on what we learned directly from Billy's performance.”

The comic-book Dr. Manhattan has white eyes and no eyelashes or eyebrow hairs, but Snyder decided to also give him these attributes after Imageworks demonstrated to his satisfaction that it could be done. His eyes were given bright-white pupils compared to the rest of the eye to create a sense of eye direction, and those were also used as a light source to help illuminate surrounding skin.

Related Links

Step by Step: Watchmen Phantasmagorical effects abound in Warner Bros.'' Watchmen, which isn''t surprising given its origin as a graphic novel about superheroes. But alongside the film''s otherworldly visual effects... Step by Step: 300 When Director Zack Snyder took on Warner Bros.' movie version of the graphic novel 300, he used wall-to-wall visual effects to create the film equivalent... Animated Performance "A lot of technology had to be created in advance." This comment from Jerome Chen, the senior visual effects supervisor on Paramount Pictures' Beowulf...

Imageworks eventually went further and added a fine coat of peach-fuzz hair to Dr. Manhattan's skin like real people have. “[This was] the kitchen-sink mentality,” Travers says. “We wanted to throw everything on him that could make him realistic, especially where skin and hair were concerned.

“After our tests, we decided we wanted subtle hair. Every shot, [Dr. Manhattan's] peach fuzz was part of the rendering process. Humans, in bright rim light, have that hair on the edge of the skin that lights up when photographed. All the time, people work on realistic skin shaders that ignore the hair on the skin. We thought it was essential, so we added it. I'd have to say the best work we did on the character are the close-up shots of his face.”

Imageworks also used compression maps for skin animation — a technique designed to illustrate tension and compression on one part of the skin compared to another. “When the skin compresses, like when lips purse, the skin becomes bumpier,” Travers says. “When tension is created, wrinkles would lessen. If you straighten your fingers, you notice the skin on the knuckles bulges up. That has to do with tension of the skin. Every shot we built, we had compression maps — bump maps that activate every time he compresses his skin.”

All sorts of other effects complete the Dr. Manhattan look — including a CG internal body mass of fluids and pinpoint light that move in concert with his body to give the impression that the character is, in a sense, a window on the entire universe. A final key ingredient in completing his look, though, was the digital-intermediate process at Company 3, Santa Monica, Calif. Snyder and Fong did the DI with Snyder's longtime collaborator and head of Company 3, colorist Stefan Sonnenfeld, whom Snyder calls “a major resource in how [they] approached the look.”

Fong and Des Jardin both say the DI was particularly helpful in smoothing out the blue glow and making it consistent.

“We always had an eye toward the DI [while shooting], making sure those shots were as neutral as possible,” Des Jardin says. “That gave us the flexibility to have Stefan push it in any direction we wanted. Once you put that definitive blue glow into the images, you establish a look or color ratio or luminance ratio that would be difficult to alter in the DI. So before we shot those scenes, we got a set of test shots into Stefan, and he and Zack and Larry played with them ahead of time, so we would know if we were in any trouble for color-correcting or not color-correcting the blue in the DI. In those tests, we found that blue is such a specific frequency that it gets into areas that you don't imagine. It gets into shadow areas and makes it look strange. So it helps to have a DP and director who have a specific idea of what they want to do visual-effects-wise in a scene. We had the LEDs, but no one had tried this before, we were always guessing on set. We had a dimmer to increase or decrease intensity, but until the camera tests, we didn't know what the response would be. It took probably a quarter of the way through the Manhattan shots before we had a handle on it. I worked with Larry to meter [the light], of course, and pick a luminance value, but to [Snyder's] credit, he accepted all of that at face value. The crazy falloff in the color was reflected in our composites. But when we took those shots into the DI, Stefan was able to isolate and control the values nicely.”

Still, Fong says the project's lengthy testing phase allowed filmmakers to shoot and light the Dr. Manhattan sequences strategically. “[That way, Sonnenfeld's work would] not be to save the movie, but rather, to be the frosting on the cake. Like anything else — if you give him something good to start, he'll be able to make things even more interesting in the DI. Stefan helped immeasurably in keeping the look of Dr. Manhattan consistent, and also making it seem like the blue interactive glow was actually being cast from the element itself.” As complex as the effects work was, Watchmen is hardly the same kind of digitally constructed movie as Snyder's last extravaganza — the entirely greenscreen-produced 300. In fact, he says he considers Watchmen more of a “normal” movie where he could focus much of his personal efforts on performance.

“I've done lots of visual-effects-based projects — commercials and 300 especially — and the big lesson for me was just being able to know how to work with the technology so it is not a hindrance, but rather something that can make the whole thing better,” Snyder says. “This time, instead of worrying about scary CGI or set extensions, I considered all that a tool and focused on performance. But it is fun as a filmmaker watching the movie evolve as we put these pieces together.”