Minority Report

Greenscreen shoot.

There's plenty of action in Minority Report, StevenSpielberg's future-noir film for Twentieth Century Fox/DreamWorks. Butin addition to having Tom Cruise do death-defying stunts, the directorwanted to reveal how Cruise's character is driven by an obsession overa long-lost son. To do so, Spielberg shows Cruise alone at night,reliving memories of his child by watching a “home-movie”that — in the year 2054 — is holographically 3D.

In one such scene, Cruise watches a “smart screen” onhis wall, which shows his son on a beach. Suddenly, the child steps outof the screen and walks forward. From the perspective of Cruise'scharacter, the boy appears to be 3D. Viewed from the side, however,it's a crude illusion — the boy is semi-transparent, trailing astreaky glow as he moves forward. On the screen, there's a child-shapedblack hole where the image of the boy used to be.

CG model.

Industrial Light + Magic was given the directive to make the boylook “sort of like a half-luminous shell,” explains CGsupervisor Barry Armour.

“It was a fine balance to make sure you know the boy isn'treally there — if it looked too real, you might think it was theactual kid,” notes Armour. “He also had to be life-size inthe projection because otherwise he'd look like a miniature.”

The ILM team decided to call upon recent CG developments thatutilize image-based rendering. Visual effects artists have typicallyused image-based rendering to construct faux backgrounds from numerousstill photos blended together, saving themselves the trouble ofactually modeling and rendering 3D geometry. But as far as Armour knew,the approach hadn't been applied to constructing a moving character. Hespoke with Steve Sullivan, a recent Academy Sci-Tech Award-winner whois versed in computer vision techniques and who had been developingphoto-modeling tools for ILM.

“My academic background was about using silhouettes fromcamera images to build discrete object models,” says Sullivan,who solved the problem of constructing the moving character for thefilm.

Background screen.

Numerous cameras needed to surround the boy to capture a usefulsilhouette, and Sullivan had to write software that would “carveout” the boy's shape from a multitude of filmed images, using theintersecting outlines to derive a 3D polygonal form. Once the 3D shapeof the boy was determined, filmed images could then be mapped onto thatshape like texture maps.

Since the holographic effect was supposed to be of“home-movie” quality, Sullivan says that “it couldlook a little messy and weird.”

“Usually you want something that's very literal andprecise,” Sullivan explains. “These techniques inherentlyhave all kinds of artifacts. If you wanted them to really modelsomething, you probably couldn't use them directly. But in this case,the artifacts were interesting, so we decided to try it.”

Finished composite.

ILM's approach involved building on stage what Sullivan calls“a cauldron of green.” This 360-degree green-screen stage— essentially the photo-modeling equivalent of a motion-capturestage — had a 10 x 10 foot area around which the boy could walk.Eight DVcams were set up — one high and one low at each corner— providing 360-degree coverage.

“Those cameras could be fairly low-res because the imagequality was supposed to look fairly amateurish,” Armour explains.“We then had three main cameras that were high resolution, turnedon their sides so that we could get a vertical image of a higherquality.”

Unlike motion capture, which produces data points that are used toanimate a CG character, this approach produced footage from the 11cameras running simultaneously.

“These sorts of techniques capture arbitrary shapes so youdon't have to put markers on anything,” Sullivan observes.“The characters can just go out there and act and do what theyneed to do.”

“As long as the kid didn't move out of the view of any camera,he could walk 10 feet in any direction,” Armour adds. “Ofcourse, it was hardly trivial to light and shoot on this stage. We hadthe biggest amount of green I've ever seen. We had cameras pointing inevery direction, so we had to make sure that the light was notinterfering with the capture of the images.”

To ensure that the cameras were rigged and synced properly for thephoto-modeling shoot, ILM motion-capture supervisor Seth Rosenthaloversaw their placement.

While Sullivan's photo-modeling software was new, the task ofblending the texture-mapped images onto the boy's 3D shape simplyrequired writing RenderMan shaders.

“It was fairly straightforward,” recalls Sullivan.“Getting the proper level of translucency for the holographiceffect was a matter of dialing things up or down.”

Sullivan notes that ILM's compositors, who use in-house compositingsoftware, then had a lot of freedom to experiment with the texturesthat they mapped and wrapped.

The finished effect, observes Armour, had many “strangelynice” artifacts.

“From the side, you can see the edges of the character goingback into the screen, like streaks or slit-scan effects,” heobserves. “Having some of the artifacts show through thecharacter actually gave it more of a three-dimensional look.”

For his part, Sullivan hopes that this photo-modeling effect issubtle enough to not distract from the poignant moment in the film.

“It is supposed to be a very personal moment between a guy andhis lost kid,” Sullivan concludes. “I hope that stillhappens.”

Director - Steven Spielberg
DP - Janusz Kaminski
For ILM:
Visual Effects Supervisor - Scott Farrar
CG Supervisor - Barry Armour
Software R&D - Steve Sullivan
Sequence Supervisor - Steve Braggs
Image Capture Supervisor - Seth Rosenthal
Match Mover - Brian Cantwell
— Compositing Supervisor - Scott Frankel