It Came from the Third Dimension
By the time Jeffrey Katzenberg announced last year that DreamWorks Animation would forevermore author all of its computer-animated feature films in a native 3D format (dubbed InTru 3D and developed in partnership with Intel), Monsters vs. Aliens was well into development as a 2D project. The feature began life as a melding of concepts between directing partners Rob Letterman and Conrad Vernon. Vernon yearned to make an animated 1950s-style scary-monster picture, while Letterman had been pushing a sort of Dirty Dozen adventure concept. Katzenberg put them together, and the resulting story is Monsters vs. Aliens.
Only well after the project went into development, however, did Katzenberg drop the bomb that the picture would have to be authored and exhibited, wherever possible, in stereoscopic 3D. The two directors, Producer Lisa Stewart, and other principals involved with the project promptly took "a collective deep breath," in Stewart's words. "We weren't sure what that meant," she says. "We were all a little afraid that he was asking us to add a tremendous amount of 3D gimmicks."
Letterman says the ensuing months were a bit overwhelming as filmmakers battled to hone their idea in such a way that 3D would serve the story rather than the other way around; learned to understand and accept a new workflow and the cultural changes related to that workflow; and labored to get everything done on time despite the fact that DreamWorks was continually retrofitting its animation pipeline for 3D while the movie was being made.
Katzenberg has long been an advocate for stereoscopic 3D as a mainstream digital cinematic experience. He convinced his filmmaking colleagues that his intention was to find a way for them to make Monsters vs. Aliens as they envisioned, with numerous action beats—normally a foe of the stereoscopic format due to quick-cutting issues—rather than the gimmick some feared. (There is one clever 3D gimmick that has nothing to do with the story. In the opening shot of the movie—which was inserted tongue-in-cheek by filmmakers to mock the notion of 3D tomfoolery—the DreamWorks logo transforms into a film burn of the 2D image melting away, bursts out of the 2D world, and re-forms itself in eye-catching 3D space.)
Thus, the DreamWorks technology team took on a mission to provide whatever filmmakers needed to execute their story while viewing it in the stereoscopic format throughout all phases of production. Their accomplishments include the development of a new blending tool designed to meld deep and shallow shots to allow for visualizing fast cuts without leaving viewers with eye strain; creation of a new 3D-viewing tool inside Maya; creation of an enhanced proprietary 3D video player dubbed "Mov Play" that permitted Avid editors to play back and watch material in 3D as they cut in one of several stereo formats; the buildup of stereoscopic viewing stations all around the studio; desktop 3D monitors for animators; and liberal borrowing (with permission and encouragement) from James Cameron's Avatar project to execute a moving virtual camera in 3D space during production.
"[Without getting too specific,] numbers did have to get shifted around [in the movie's budget] to support the studio in encompassing a new 3D pipeline, and some of that was a guessing game since we had never done this before," Stewart says.
Katzenberg estimates the implementation of these changes added about 10 percent to the cost of making one of DreamWorks' CG movies today.
"When anything new comes along—and in this case, everyone involved had to adapt to a new language—there is a learning curve to it," Katzenberg says. "Then you get to the other side of the learning curve, and you have to maintain the quality and efficiency of the work, so all those costs go up exponentially. Right now, it's more expensive [to author a CG feature film stereoscopically] than to do it with live action. But for us, it's more exciting and creates a movie-theater experience that is special and unique, beyond what consumers can see at home and different even from [films traditionally produced in 2D and then converted to 3D]."
Monsters vs. Aliens is not history's first feature-length digitally animated movie authored stereoscopically for cinematic exhibition. That distinction goes to 2008's Fly Me to the Moon, which was produced by nWave Pictures and then exhibited in the IMAX (70mm film) format. Monsters vs. Aliens is, however, the first major studio CG picture made this way. More importantly, it's the first made this way as part of a permanent stereoscopic initiative. It's also the first authored natively in 3D and exhibited in the RealD 3D format in digital cinemas. (It will also be seen in selected IMAX theaters.) It was then converted to 2D for exhibition in nondigital cinemas. Such projects have traditionally been created using 2D processes and then converted to 3D, but DreamWorks opted for the opposite with Monsters vs. Aliens, and it will continue that approach going forward.
As a result, the creation of Monsters vs. Aliens from previz through final rendering as a stereoscopic project, along with the corresponding movement of the DreamWorks pipeline into the 3D forest, can be viewed as a significant industry development. It has the potential to be as influential from a technical point of view—in terms of bringing stereoscopic production into the mainstream—as the 2007 U2 3D concert film was and as Cameron's Avatar project is expected to be when it finally debuts.
Phil McNally, the studio and movie's stereoscopic supervisor, points out that DreamWorks was well-suited to the effort because Katzenberg had been pushing for the studio to experiment with 3D for years.
"DreamWorks has been doing 3D tests on almost every animated film it has made, waiting for the time when it made practical sense," McNally says. "The IMAX film Cyberworld  has the bar scene from Antz rendered in stereo; the Shrek 4D ride at Universal Studios is stereo; and we did various tests on Shrek 3, Flushed Away, and others. So none of this happened overnight. In my opinion, we have always had the desire and good ideas about how to do it, but until the arrival of digital cinema and the implementation of the RealD [system] in theaters, it was never totally viable. Now it is."
McNally has extensive experience on the front lines of stereoscopic animated feature work, having worked at Disney when that studio was converting Chicken Little, Meet the Robinsons, and Tim Burton's The Nightmare Before Christmas into some of the industry's first stereoscopic presentations of CG animated films. This time around, however, he says lens choices, composition, and editing style during production were all informed from the get-go by the stereoscopic format's parameters rather than by a traditional presentation of a flat, 2D world.
"Cutting speed, lens choices, action beats—we are dialing all these things in for 3D each time," he says. "Each review with directors—it's all in 3D from the beginning, so we are literally making it by seeing what we will get in the end as we go along. When [Katzenberg] announced we would be doing it in stereo, the shockwave went rocketing out across all departments. Fortunately, we had done those tests in the past, and so it wasn't as though people didn't understand stereo. But, still, this was the first time a studio had committed to this for every show. So a big part of my job at the start was working with the studio to figure out what tools to put into place so that editorial could see their work in 3D, do reviews in the dailies room in 3D, see inside Maya in 3D, and so on. The main thing, at first, was getting the dailies room set up so that we could represent the stereo view you see in public. Right now, we have three rooms [on DreamWorks' Glendale campus] that are fully up to the [DCI digital cinema specification]—with full DCI-compliant projectors for the RealD system in two of them, and one for the Dolby [3D] system—and we are now looking at putting in a fourth, as well."
One of the earliest moves the production made was to accept an invitation from Cameron to visit the Avatar virtual set and research that project's advances in terms of virtual stereoscopic cinematography.
"We had a lot of sharing back and forth with Cameron's group at that time since we were all trying to get 3D going at about the same time," Letterman says. "I distinctly remember asking Cameron how he deals with what I felt were very tricky constraints imposed on the filmmaker by 3D, and he basically said, ‘Don't worry about that stuff. Just shoot your movie.' From that moment on, it was a big relief for me—we allowed it to become a natural process for us. We had an action movie, and rather than altering that, we asked, for instance, for tools to allow that kind of pace."
McNally says that the production's interaction with Cameron's team was particularly helpful.
"The main thing we took away was [how to do] virtual camera work on a CG set," McNally says. "Not just the stereoscopic nature of it, but the idea that we have a piece of technology that looks like a small TV with handles on it. And when you look at the TV, what you are really seeing is the virtual CG set [as you walk around the set]. As we move it around, the motion is recorded as animation curves in the computer. To be able to wander around the set as if you were holding a camera is something that Damon O'Beirne, our [head of layout], was very excited about—the idea you could reshoot your animation with a handheld, live feel. This was especially fortunate for some of our action sequences where you want to feel you are under pressure with things exploding and ripping apart. It's also allowed him to do very subtle adjustments—the kind of things that only happen when there is someone operating a camera. It's pretty easy in CG to make camera moves smooth and almost ahead of action, predicting where things would move, because we can dial it in so perfectly. Adding a little bit of imperfection is a good way to get past that. So we set up our own camera capture system at DreamWorks after seeing the benefit of it on the Avatar set. It's really a way to look at your virtual environment in the computer if you are standing and moving through it. The main benefit is the natural feel of the camera that it gives us."
Among the action sequences McNally references is the movie's centerpiece action moment: a battle between monsters and aliens in downtown San Francisco and on the Golden Gate Bridge. In terms of both visualizing the sequence and cutting it, filmmakers insist the scene achieves their pacing goals without bowing to the traditional confines of 3D, but with all of the medium's advantages.
Letterman says the scene is the one he is most proud of in the movie and suggests that it has a visual depth and style that would not normally be found in a standard CG presentation.
"[The scene] is meant to have the feel of live action, and that means it can't be too perfect [with camera movement] as most animated movies tend to be," Letterman says. "They also tend to be too linear. Our approach with this scene was to break that up and do something totally different, including the way the camera moves through space. There was specific intent to move straight down the Z axis, not moving across the screen as much as moving behind the screen. We put in handheld shots and quick cuts and racing along the bridge, and we also shot coverage, which is uncommon in animation. In this sense, 3D really stood out because we could go down deeper and then come back instead of just pushing out at the [viewer's] face. There was a sense of volume to it—moving through very big spaces. We tried to make you feel the volume, and those are choices we probably would not have made in 2D."
Among the project's greatest challenges, though, was figuring out ways to ensure viewers did not experience eye strain during sequences such as the San Francisco battle. The idea was to allow the film's editors, Joyce Arrastia and Eric Dapkewicz, to largely cut those scenes with a pace similar to how they would do it in 2D. The 3D technology team would then apply a blending process to transitions between those shots to reduce the eye-strain risk.
"[The blending process was] an extra process on top of other processes as we checked the work all along the way," Vernon says. "They came up with software that let us go shot by shot, blending a deep shot with a shallow shot to lessen that straining sensation. It was an example of how technology moved forward on this project so that we could have a solution for how 3D affected what we would normally do on a typical animated film."
McNally largely handled the stereo-blending process himself using the new proprietary blending tool—also called a stereo perfection tool—created for the project during what he calls the added-depth grade, or stereo-blending stage—essentially a quality-control process to ensure action shots behave during transitions.
"You don't want your eyes jumping too much from a close-up of a robot to a monster on the bridge," McNally says. "I adjusted the stereo to blend the depth during the transition from close shot to distant shot. The distant shot might be set in stereo so that the robot is close for the first few frames, and then we gently ease it back into position to give your eyes an easier, softer transition even though it still feels like a hard cut. I graded those shots from a depth point of view, capturing shots that needed adjustment on a fundamental level and blending them together. The software reads all the shots in the sequence, but also gives me access to the camera curves that control the stereo, letting me manipulate them in realtime. This was an important step in eliminating the kind of eye strain that has been problematic with 3D in the past."
Another innovation was the creation of a stereo camera rig in Maya to view native stereoscopic imagery, according to McNally. He says inhouse stereoscopic software developer Paul Newell worked with Autodesk to develop a software camera that has a left-eye component and a right-eye component and automatically calculates stereo settings, which was helpful during daily animation reviews.
"It was developed from an artist's point of view," McNally says. "If you view a shot with one point being your nearest point in the scene and the other being the farthest, and you want them to appear at a certain place in relationship to the audience in the theater, you are really talking about pixel shifts. If you view a 3D image without glasses, those little separations along the double image's edge are a certain number of pixels apart. This way, we came up with a Maya stereo rig that has the ability to set the near plane, the far plane, and tells the camera this is the stereo space I want to work with. Then the artist puts in the shift values they want to achieve in the theater, and the software calculates the interaxial distance [distance between the two separated lenses] and the zero parallax setting [convergence point] based on what we requested. This way, we could say in dailies, ‘Give it five more pixels of depth,' or whatever the situation was. It created a common language of depth we could all discuss."
DreamWorks focused on giving artists the ability to see 3D depth easily while working throughout the project. This was done at various viewing stations that were essentially high-end consumer-level DLP monitors and on their desktops, where they typically were set up with CRT monitors and stereographic active-viewing glasses. Currently, however, McNally says that DreamWorks is in the middle of transitioning away from CRT technology and toward passive, polarized Hyundai 3D LCD W220S monitors—particularly 24in. models that work well with RealD polarized 3D glasses. The studio expects to have them in place in time for production on the upcoming 2010 3D feature, How to Train Your Dragon.
Still, throughout production, there was the looming question regarding the fact that more than half the screens showing the movie to consumers will be, at least for now, 2D screens. The exhibition industry's conversion to stereoscopic presentation technology has not kept pace with Hollywood's 3D production slate (see sidebar for Katzenberg's view of this transition). Therefore, some subtle changes had to be made in terms of transitions and other kinds of cuts.
However, the filmmakers say they ended up with a richer 2D experience largely because many of their typical cheats—such as the use of animated matte paintings—were out of the question for a 3D-authoring process, and their attention to fine detail generally had to be greater because of the nature of the format. However, they concede it is not a typical 2D experience merely by virtue of the fact that their decision-making process in many areas had, by definition, to be different than if their primary authoring concern was a 2D presentation.
"I think [3D authoring] made the 2D version more cinematic and therefore higher-quality," Katzenberg says. "That's simply due to the level of attention and detail that has gone into the approach to the cinematography on the picture. We've really never had so many resources, creative resources and technical resources, on thinking through every single shot on a film. That will obviously come through in the 2D version of the movie."
As Monsters vs. Aliens got ready to debut, DreamWorks Animation CEO Jeffrey Katzenberg took time to chat with millimeter about the movie's significance as the launching point of DreamWorks' agenda to convert the animated film industry to the gospel of stereo. Katzenberg has been promoting this agenda for years and has been aggressively pushing the exhibition industry to accelerate its conversion to digital cinema generally and to stereoscopic-capable systems specifically. He discussed the state of that transition, why it's important, and his passion for this unique format.
millimeter: How are you feeling about the debut of Monsters vs. Aliens in terms of how it will impact DreamWorks' agenda with stereoscopic filmmaking moving forward?
Katzenberg: For DreamWorks Animation, it's our first opportunity to actually author a movie using these new digital tools and show the world what we can do with them. I'm feeling pretty confident about it. As you know, we committed all our productions from here on to being authored in 3D, so in that sense, I think it's a beginning of a new era in movie-making. Of course, everything we do in 3D won't matter if we don't tell a great story—that remains at the root of it all. I think we achieved that here, and now we have another opportunity, another device, for telling that story in a particularly exciting way. I think our audience will find [the stereoscopic presentation] an exceptional and unique way to see a movie.
What is the state of the conversion of cinemas so that they can present this movie and others like it in all their stereo glory? The cinema owners group (DCIP Consortium) only agreed late last year to a framework for wiring cinemas for stereo, and since then we've had a severe economic downturn. What's your view on where that transition stands?
The economic crisis has had a dramatic impact on the rollout. Right now, we are anticipating there will be approximately 2,000 screens in North America that can show this movie in 3D, and that includes about 150 IMAX [large-format film] screens. That's about half of what I originally anticipated this time last year, before the financial markets collapsed. So for sure, there has been a meaningful delay in getting this as widely deployed as I hoped and wanted. But, having said that, to be on 2,000 screens is more than enough for there to be a real proof of concept for this new movie platform. I would say that I anticipate that by the time Shrek 4 comes out next May, we hope to have around 7,000-plus theaters able to play it [in 3D].
For you personally, why have you been so passionate on the topic of the cinematic 3D-viewing experience, and what have you learned trying to sell it to the industry and to consumers? The Monsters vs. Aliens Super Bowl promotion, for instance, was criticized by some media pundits for not living up to the hype.
We did a stereo test with a sequence of Kung Fu Panda as a proof of concept for our artists and to help us develop the tools we needed to make a whole film this way. But that test sequence in 3D was also valuable and instructive for convincing exhibitors and various partners around the world so they could see what we were going for with Monsters vs. Aliens. I think they saw this isn't their father's 3D.
There is so much going on with [home-viewing technology], but I still feel that what a movie theater can do on a very big screen, which impacts peripheral vision and is important for that sense of immersion, in a completely black environment, allows us to deliver a super-high level of 3D resolution. So, I believe in this because it is truly a movie-theater experience that is special and unique and beyond what you can find somewhere else.
Do I think we've done everything perfectly along the way [developing and promoting the concept]? No, I don't. Do I have any regrets? No, I don't. Do I think this is something that everyone will embrace as the second coming? Obviously not. But I am extremely excited about the quality of the experience we are making available to movie-goers.
What about the role of the technology companies in making all this happen—helping DreamWorks build a real and permanent stereo pipeline?
Without HP and Intel, we could not have been able to get to where we are today. We don't have the resources and an R&D staff big enough to conquer the many unknowns we've encountered on the way to making this movie. Without those companies, we simply could not be making these movies. And that goes above and beyond the traditional vendor-customer relationship. They extended themselves so much in so many ways. In some ways, I feel both of those companies are the godparents of DreamWorks Animation and the work that we do.