Spider-Man

The shot begins on the real face of actor Tobey Maguire, hereincorporated with the body of the wire-frame CG Spider-Man.

The promise of virtual cinematography is being able to move a camerain gravity-defying ways. That's what director Sam Raimi wanted forhigh-flying scenes in Columbia Pictures' Spider-Man. Visualeffects designer John Dykstra explains, “The idea was thatSpider-Man has his own personal cameraman, like skydivers havecameramen who are part of their choreographed routines.” Toachieve this, the visual effects team at L.A.'s Sony PicturesImageworks needed to create convincing synthetic elements for thisvirtual camera to “photograph,” including acomputer-animated Spider-Man and the photoreal environments throughwhich he flies.

In a scene that's essentially all digital, but which has to cut intoa photographed sequence, Spider-Man stands atop a 20-story building andlooks down at the street below. “The shot starts close on hisface,” notes Scott Stokdyk, Imageworks' visual effectssupervisor. “Then it does a 180 [degree] camera move around himwith the camera rising upward.” By the time this 15-second shotends, Stokdyk explains, “You're looking down at the back of hishead and seeing the building he's standing on as well as the buildingacross the street and the traffic on the ground. Then you see him leaptowards the building across the street, holding onto his web. It's areally slow camera move, so there's lots of time to examine Spider-Manand the environment around him.”

Fully 3D-CG interiors were created for the rooms that Spider-Manwill fly closest to.

The shot begins close-up on actor Tobey Maguire because the directorwanted to show him as a fledgling superhero unsure of his powers.“To pay off that idea,” says Dykstra, “we needed tohave the actor's face. Against greenscreen, we made a move around Tobeywhile he was giving us the necessary expression. Then we crafted a CGmodel to fit around him. It was a virtual character with a realface.”

“We basically match-moved the camera move as we went fromlooking into Tobey's eyes to the point where we're three-quartersbehind him, and we don't see his face anymore,” Stokdyk explains.“We sort of eased into a completely virtual camera at about eightseconds into the shot. For tracking we used Maya and traditional handtracking along with our own plug-in tools. It was a pretty exactingtrack because of the soft surfaces of Tobey's face.”

The CG rooms were composited within the building in the finalshot.

The close camerawork necessitated that the character's CG clothingbe crafted in great detail. “The concept was that since it'searly in the movie, Spider-Man has a homemade costume, like a teenagermight piece together,” Stokdyk observes. “He's wearingsweatpants, a long-sleeved T-shirt on which he's spray-painted theSpider logo, and a ski-mask-type hood. Even when we're close-up on him,his costume is CG. We created the material using cloth simulation withMaya Cloth and a suite of plug-in tools developed here. We did a lot ofwork to get details like the little pilling that you see on T-shirts.Since it's pretty loose fitting, we also added wrinkles — some ofwhich are geometry from our cloth simulation as well as additionalwrinkling from painted displacements. Depending upon how he moved, wemoved both the geometric wrinkles and the displacement wrinkles.”The costume was rendered using RenderMan.

The shot ends high above Spider-Man as he prepares to leap off thetop of a building.

Because this shot contains a dynamic 180-degree camera move —with all the attendant perspective shifts — it was necessary thatmuch of the environment around Spider-Man be created in 3D-CGI.“We surveyed and took reference photos of about 15 hero buildingsin New York City,” says Stokdyk. “Then we modeled them fromscratch in 3D using Maya. We also set up a building system in Houdiniusing plug-ins written here that enabled us to efficiently render allthis geometry. Because there's so much geometric detail in this city,we used Houdini's instancing capabilities and its nice hook intoRenderMan to handle all that geometry.

“The building that Spider-Man swings toward needed to be 3Dbecause we get pretty close to its interiors,” notes Stokdyk.“You see parallax shifts on beds and lamps and dressers. In thecases of the biggest rooms that he flies towards, we built and texturedfull 3D environments inside them. But for the rooms that we don't seeclosely, we did some tricks — kind of like those real estatewalk-throughs on the Web. You take a fisheye lens that allows you tosee the whole scope of what's inside a room, and then you map that ontofive sides of a virtual box. Once you line up the walls, you can seethe walls moving relative to each other and doing little perspectiveshifts as you fly by.”

Essential pieces in this jigsaw puzzle of a shot were the streetelements needed to make the scene seem alive. “We texture-mappedour streets using projections of real streets and nighttime platephotography,” Stokdyk says. “Then we added CG traffic,trees, newsstands, streetlights, and people.” While theSpider-Man character was keyframe-animated in Maya, the CG people onthe street were motion-captured at Venice, Calif.-based House of Moves.HOM used a Vicon 8 camera system, and Imageworks created a plug-in toMaya that accepted the raw motion capture data and applied it to itsskeleton capture system.

To complete the background cityscape, Stokdyk explains, “Weused photographic elements that we seamed together to create a virtualsphere of tiles around the whole shot. We set up our virtual tiles in3D space in Maya and then exported that to our inhouse compositingsoftware Bonsai, which took those 3D tiles and stitched them togetherin 2D in the final composite.” Finally, the image was desaturatedto match the look of the photography in the rest of the sequence.

That integration was crucial to making this shot credible.“This long shot appears in a sequence that is fairly quick-cut,but such a big pace change was needed to show Spider-Man'strepidation,” Dykstra observes. “Of course, without thiscogent story point, such a gee-whiz shot would just have been effectsfor effects' sake alone.”

Director: Sam Raimi
Director of Photography: Don Burgess
Visual Effects Designer: John Dykstra
For Sony Pictures Imageworks:
Visual Effects Supervisor: Scott Stokdyk
CG Supervisor: Dan Eaton
Animation Supervisor: Anthony LaMolinara
Spider-Man animation: Jeff Lin
Match-moving: David Spencer
Costume shaders: Sing Foo
Houdini plug-ins: Steve Lavietes
Maya motion-capture plug-ins: Alberto Menache