Developing Drop

Before and after skyscraper collapse from 2012.

Software tools for visual effects include such hall-of-fame names as Autodesk Maya, Side Effects Software Houdini, and Pixar RenderMan, but with each passing year, VFX studios worldwide are expanding the standard toolkit to meet increasingly ambitious production challenges. A recent case was Digital Domain (DD) in Venice, Calif., which was tasked with demolishing Los Angeles for director Roland Emmerich's apocalyptic 2012. The software team at DD knew that conventional methods previously used for simulating rigid body destruction in Houdini would be insufficient to handle the scale of disaster that Emmerich envisioned. The number of collapsing buildings was too large, and the director wanted the freedom to direct how the simulations appeared to camera. DD would need to come up with a more efficient and flexible method—and do it fast.

"We may have had about three or four weeks," says Nafees Bin Zafar, a software engineer with nine years of experience at Digital Domain who worked on Pirates of the Caribbean: At World's End as well as 2012. Zafar, who is now at Dreamworks Animation, had previously earned a Scientific and Engineering Award from the Academy for DD's fluid-simulation software, so he was as prepared as anyone could be for developing a new sim strategy on deadline. "We had a team of engineers and artists that had worked together before," he says, "so we had confidence. We had a good sense of how to build artistic controls on top of the math. That impacts workflow, and how the software appears to users."

While DD's expertise with water simulation provided some precedent, Zafar says, "The math of rigid-body sims tends to behave closer to reality than water-sim math does. With rigid-body sims, if you have a reasonably sized object and it falls at a real-world rate, it's a little more forgiving. Rigid-body sims look more like traditional animation."

Along with Zafar and his fellow Sci-Tech winner Ryo Sakaguchi, DD's development team included Michael Clive, Ken Museth, Ramprasad Sampath, and Marten Larsson. Computer Graphics Supervisor David Stephens, who had just joined DD from Sony Pictures Imageworks, was also a key player. He had overseen destruction scenes at Sony for Watchmen, so he understood the challenges presented by 2012.

Rather than try and invent a new approach from scratch, the DD team considered building on videogame technologies such as Havok and Nvidia PhysX.

The studio had previously used the Unreal Technology game engine for commercial productions, so there was some familiarity. Stephens notes that more and more software companies are heading in the direction of using game physics, citing tools such as RayFire. But DD ultimately settled on using the open-source physics library called Bullet as a foundation for its rigid body simulation approach. The team then added techniques for shattering objects that were made of varying materials, such as wood or glass.

"It's been our experience that open-source projects help a lot," Zafar says. "Because typically in feature effects we end up needing to work very differently than the original intentions behind libraries that were built for games."

Using Bullet as a foundation proved to be a savvy choice. "When we ran into conceptual problems or didn't know how to get something done, we posted questions to the Bullet user group," Clive says. "People knew that we were incorporating Bullet into a Houdini-based pipeline for film effects. Usually we got feedback, including from the author of Bullet, Erwin Coumans, who's become a good friend of ours."

Drop test: multistory shaded model collapse

The realities of crashing buildings for 2012 put the new tool—aptly named Drop—to the test. Stephens explains the process. "We'd start off with something in a Maya scene, where an animator might indicate where a building should start to tip," he says. "That would be the first part of the puzzle. We'd have all the constraints binding different parts of a model together. The model would be effectively 'slaved' to that Maya animation just to get it going. At some point, the simulation would be switched to live, but only in pieces—so that parts of the building would be simulated and others were still under a more kinematic influence. Eventually we'd also animate the constraint strengths between pieces so things could crumble more on cue. By an artful use of those tools, we were eventually able to art-direct, more or less, what a simulation would look like. You'll always have a certain amount of randomness—that's just the luck of the draw—but we've basically stacked the deck as best we can to get the timings and the basic blockings right. Then we can iterate to finally find the exact-right simulation.

Related Links

2012 Step by Step Director Roland Emmerich was famously destructive in the civilization-in-peril movies The Day After Tomorrow and Independence Day, so when he called 2012 the biggest visual effects movie he's ever done, that's saying something... Water, Water Everywhere If the Academy of Motion Picture Arts and Sciences' Scientific and Technical Awards had a mantra this year, it would be, Water, water everywhere and not... Driving Force Software spawned by the needs of production studios is a venerable tradition in the animation and visual-effects industries...

"Directable chaos is the name of the game. You know you want something to fall a certain way with a certain timing. But if the director wants it 4ft. closer, that's a directable goal. At that point, your goal is to get the right physics look to it—but the fact that you can influence it to hit a mark is the controllability part."

Control was one goal for the team on 2012; speed was another. "Speed is the principal thing we have to accelerate in all simulations," says Clive, who previously worked on DD's simulations for Pirates 3 and The Mummy: Tomb of the Dragon Emperor. "With speed, you get artistic control and quality. You can have the world's best sim, but if it takes 40 hours to run, you won't be able change anything. You'll get stuck in a loop with a 40-hour waiting period. With Drop and the technologies associated with it, you get the ability to have direct control over what's occurring in the simulation, which is really valuable from an artist's perspective. If you don't feel you have that control, you won't be able to address the notes you get from the director. You won't be able to make that corner of a building fall four feet closer because it will take 20 hours to generate that. With Drop, it's 45 minutes."

"One of the nice things was that we were able to do sims in layers," Stephens says. "For example, we could have a couple of hero buildings destroyed and then use those sims as collisions for the next layer of collapsing buildings. If you tried to do that with miniatures, you'd have to have 15 to 20 buildings all collapse on cue somehow to get a directable performance. That's obviously not going to happen. But with this approach, you can work from the inside out. You work on the ones you know you want to make the hero buildings of the shot and then have all the other sims work off those. And if the director thinks you blew up too many windows, you go back and re-sim it, leaving more windows intact. You don't have to re-run the original building sim; you just have to re-run the secondary sim. The ability to layer these sims gives you enormously powerful artistic control."

Drop test: shaded model house collapse

"Another aspect of Drop that is pretty neat is that we're taking for granted that stuff is fractured," Zafar says. "It doesn't come out of modeling that way. There are a bunch of tools that perform the artisticially driven fractures into the correct shape. Then it goes into a phase that rigs them up in custom ways and the rigging reflects the kind of materials they're made of, whether it's rebar or concrete. The behavior is different, and the rigging tools take care of that. Perhaps that's where most of the novelty is—how the fracturing and rigging is done. If you look at a Drop setup, there's really just one simulation node. There are a bunch of other Drop nodes that do all the other stuff."

"The entire architecture was designed to save the most amount of work to be done in the last step," Clive says. "Everything that you want to do that's quick and easy you can do as prep work. If you were a cook, it would be like dicing onions and putting them in a bowl until it's time to use them. You do your prep work in a fast, lightweight way, and then you throw them into the heavy sim. So it doesn't slow you down. This wasn't just used for buildings—we had an open space sim where we ended up making car rigs with turning wheels and bending axles. We even figured out how to make trees bend and shimmy during an earthquake."

The end result was an extensive library of simulations available to the production team of 2012. "We had some airport buildings that had very large structures that we built libraries out of—we would run these sims out for 1,000 frames or more," Stephens says. "A couple of those would cook overnight, so we then had thousands of frames of extremely detailed buildings in all stages of collapse. We could then slug them in anywhere."

Digital Domain's development team says that the Drop system created for 2012 reflects an industry trend. "We're getting the technology to a point where we're freeing artists from having to be technicians," says Stephens, who is currently working on Tron Legacy. "At the moment, Drop is still very technical to use. But over time, I think we'll end up seeing more intuitive interfaces. And as machines get faster and algorithms get better, the time of iterations will go down as well. We'll continue to see this approach bloom. There's no going back."

See Digital Domain's Reel-Exchange profile and reel