Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now


Legato’s Laboratory

Rob Legato ignores the African elephant lumbering past his Ford SUV during the brief ride up a dusty mountain road leading from “Africa” to “China.” Legato wants to get an Eymo camera rig placed and ready to shoot pickups of the SUV’s wheels bouncing along the dusty road so that he and his crew can move on to “Greece.”

For Rob Legato”s Ford SUV spot, the crew filmed one elephant and zebra and digitally cloned them in post.

This summer, Legato spearheaded the creation of locations from all three countries on a searing hot mountain in Simi Valley, Calif., for a Ford SUV commercial he’s directing for production company Untitled Films dubbed “Ford World Traveler.” He’s hoping to wind up principal photography today, but the Eymo rig is delayed and crew members still need to dig up parts of the road to make the SUV’s wheelwell footage appear seamless. Five men chop at the dry, cracked earth as Legato is told the Eymo rig is still about 15 minutes away.

Although Legato is an Oscar-winning visual effects supervisor, he’s determined to film most of the commercial as practically as possible, and then complete the illusion of a Ford SUV traveling the globe using a variety of 2D effects. That’s why he hired longtime collaborator Matthew Gratzner to build a model of a portion of the Great Wall of China in a nearby parking lot rather than creating a CG version, and why he trucked in the elephant and a zebra.

It’s also why he’s editing and largely finishing the piece himself in the basement of his Pasadena, Calif., home — a place that Legato has transformed into something of a laboratory for his preferred “handmade” method of filmmaking. It’s an approach based on his personal preferences, improvements in HD post technology, a variety of partnerships with vendors such as Adobe, and the positive vibes he enjoyed last year helping Martin Scorsese make The Aviator (see the January 2005 issue of Millimeter for more).

Legato is making the Ford spot at the same time as he’s prepping a new digital facial replacement methodology for an upcoming PacificCare commercial, production of which will ramp up as soon as he finishes the Ford shoot. The two spots are the first commercials Legato has directed in more than a decade.

Legato (far right) with producer Ron Ames (far left) and DP David Darby (center) on location during production of Legato”s Ford SUV commercial.

The Philosophy

In combination with his concurrent work prepping visual effects and directing second unit for Scorsese’s next film, The Departed, and the 3D previz and performance capture techniques he’s researching for Jim Cameron’s upcoming 3D movie, tentatively titled Project 880, the spots are also proving helpful to Legato’s ongoing preparation for his own feature film directorial debut. Legato says he is currently mulling over projects and expects to decide in the next few months on a movie to call his own.

But, from an advertising point of view, his commercials are also designed, among other things, to prove that a feature-film-like postproduction sensibility aimed at producing extremely high-resolution, high-fidelity, rich colors can be applied to commercials in an economical way. He’s also hoping to use the spots to R&D various feature film processes.

“I’m directing commercials and helping [Scorsese] and [Cameron] with production enhancements on their films, rather than taking on a big effects movie [as a visual effects supervisor], and that’s a strategic decision,” Legato explains. “What interests me more right now is a film I can largely do myself, finish myself, do it inexpensively, and still have it look like a bigger, more expensive film. On these commercials, we’re learning to finish the imagery as we might a feature film. I want to get rid of that disconnect between shooting, editing, and finishing.”

That said, Legato is not promoting the end of crews, vendors, or big budgets. In fact, he’s working with Venice, Calif.-based visual effects boutique Digital Neural Axis (DNA), on the Ford spot. But he is hoping to avoid the “too many chefs in the kitchen” syndrome by being able to physically previsualize and, often, finalize most changes himself, or in collaboration with small, local boutique facilities, as soon as he thinks of them while a project is moving along, rather than having to guide each change laboriously through a maze of layered hierarchies.

“It’s a philosophical way of working — to handcraft it all myself,” he says. “If I can color correct or edit it all in my basement, then why shouldn’t I? Why not avoid those additional layers? If I’m doing it myself, I can decide and make changes at the same time, and, since I’m doing much of the work myself, the changes can be implemented more affordably.”

Legato says, therefore, that the Ford and PacificCare spots, along with his current R&D work for Scorsese and Cameron, are concrete tests for his general theories. “When all those projects are done,” he says, “there will be hard, filmic evidence for this method of working — evidence that will help me sell this approach for other projects. Commercial work, in particular, represents a unique opportunity to try out complete ideas in concentrated form, and I’ve found that advertising agencies are open to trying these new methodologies if you can prove they will get the story point across.”

For the “China” sequence of the Ford spot, Matthew Gratzner built a model of the Great Wall of China.

The Basement

The infrastructure for this approach has been evolving since Legato’s early days on The Aviator. Most often, only Ames and assistant editor Adam Gerstel are found toiling in the basement alongside him. Since Aviator, they have helped him build a sophisticated Sony HDCAM-SR-based assembly line to allow him to previsualize, edit, color correct, composite, master, and render high-end imagery by himself, if necessary.

“HDCAM-SR is the best mastering medium out there right now in a portable way,” he explains. “In most cases, it becomes my original, digital negative. For the PacificCare spot, for instance, we got the source material from CBS [clips from the original I Love Lucy show] already mastered to SR tape, and that meant we had no reason to go back to film. Everything is as clean as possible in a portable storage medium. Instead of having everything on a giant server and moving it to another server, I can store it all on tape, access it with an EDL, bring it online instantly, and save the expense of needing a big facility with a big I/O department to dump everything on and off of servers.”

The basement’s basic workflow, according to Ames, allows any film footage scanned to uncompressed 4:4:4 HD resolution in Cineon log density space, or any digitally acquired footage, to come into Legato’s network on 10-bit, 4:4:4 SR tape, and, from there, to be cloned for protection and then down-rezzed for offline editing on Avid, Apple Final Cut Pro, or Adobe Premiere Pro systems. Only the SD version gets a preliminary color-grade pass, while the HDCAM-SR version stays raw.

“We use Premiere Pro on an HP [xw8200] workstation, outfitted with a Blackmagic Design [HDLink] uncompressed 4:4:4 HD capture card and a Huge Systems media vault array to digitize the SR tape, and we can then auto-conform at 23.97fps based on the standard-def offline EDL,” Ames explains. “We’ve configured an [LCD HD-capable] viewing monitor that is capable of displaying the files loaded with 2D or 3D LUTs for accurate color display. The whole system is calibrated to linearize the files and display them according to the working color space.”

Eventually, after color correction is performed on the SR images upstream from the viewing LUT and visual effects and opticals are incorporated into the cut and color corrected, Legato and his team can render and record the imagery back to SR tape as an HD master for video deliverables, or they can extract sequential .dpx files from the rendered .avi files and export them to FireWire drives for color tweaks and a final filmout at any facility filmmakers might choose.

For the Ford spot, Legato had Technicolor Creative Services (TCS, formerly Complete Post), Los Angeles, color correct Digi Beta dailies for the agency, while a pristine HDCAM-SR digital negative was created in log space, with no color correction at all, for mastering.

DNA then created various 2D effects and matte paintings and added a range of rotoscoping and other compositing tricks, such as cloning the elephant and zebra. Most of that work was performed in After Effects (6.5.1) and Photoshop CS2. What was innovative was the pipeline that Legato designed, permitting all parties involved to work in the highest possible log space all the way through the process, according to Darius Fisher, president of DNA.

“The postproduction workflow was somewhat experimental [for a commercial] in that, on one level, it was very similar to a visual effects job on a feature film,” Fisher explains. “The similarities being that we were working with RGB 4:4:4 log space footage, rather than footage that was already color corrected, or which, at least, had already had a flat pass. We were dealing with true log space images, as we would usually do on a feature film, but rarely on a commercial. Also, we were working off HDCAM-SR transfers of the uncorrected footage — essentially the film was transferred to SR tape so that we could conform from the offline to the online and have the full range of the negative retained in the resulting captures. That really helps in terms of compositing — we had far more latitude than we normally would on an average commercial.”

Fisher adds that DNA used a Blackmagic Design-based hardware and viewing system calibrated to be identical to the one in Legato’s basement. Completed effects shots were soon sent over to Legato on FireWire drives, and he then assembled the final version in Premiere Pro before sending it over to TCS for color correction and mastering to HDCAM-SR tape.

“The main point is: We shot film, but never had to go back to film again, and we could still get the final color correction done at full latitude, which you usually don’t get to do on commercials because the final conform is normally done on top of a previous color-correction pass,” Ames emphasizes. “There is an efficiency to working this way on commercials because it saves us from having to commit to final color decisions earlier in the process. Once those decisions are made, you normally have to pay a lot of money late in the game to fix it if the agency wants a change. This way, we bring a fully composited commercial that retains full color range of the original negative into the color-correction suite.”


The PacificCare spot, however, while also largely processed through the same pipeline, was a much more complicated affair because it required digital manipulation in the form of facial replacements. The Deutsch Agency licensed the moving images of deceased actors William Frawley and Vivian Vance, as their characters Fred and Ethel Mertz from I Love Lucy, and put modern dialogue about PacificCare into their mouths.

Legato instantly discarded the popular warping method of facial replacement as not sufficiently believable, and instead opted for a markerless facial capture approach, re-targeting portions of new actors’ faces to the images of the original actors. The technique partly came out of performance capture R&D work Legato is currently doing for Cameron’s Project 880.

“I just didn’t want to be cavalier and say, ‘fix it later,’ and just shoot some lips and jam them onto the faces [of the original actors] in post,” Legato says. “I wanted to break it down and make sure the facial movements corresponded to the body language. That meant largely reconstructing the faces. But I wanted it correct up-front, rather than fixing things we had already shot. So we are assembling the commercial in my basement, relying on markerless facial capture data [from Image Metrics in England].”

The basic technique began with Legato using After Effects to gray out the faces of the Fred and Ethel characters in the original pre-edits of the I Love Lucy footage.

“This let me mute the face to see more clearly how the body movement and gestures would match up with the PacificCare script,” he explains. “I then used After Effects to composite multiple shots together from the original source material, to get Fred and Ethel moving in a way that matches the script.”

Modern actors with physical resemblances to Frawley and Vance were then chosen during a casting session that Legato says resembled a looping session — new actors speaking the original dialogue from the Lucy episode as if they were looping their lines into the original footage. Then, finalists were asked to speak the new PacificCare dialogue in concert with the grayed-out faces version of the original images playing out to allow Legato to further match body movements to dialogue.

Then, Legato conducted a previz shoot, taping the actors using Panasonic’s AG-DVX100A 24p MiniDV camera on a set built by Gratzner’s New Deal Studios, designed to match the original Lucy set and painted in gray tones to maintain unified grayscale. He then edited and squeezed down that footage to select timing that worked, and returned to the stage to film the actors with a Sony HDW-F900 HD camera, capturing the actors against black backgrounds, with their heads and eyelines matching those of Frawley and Vance.

“The prep shoot really helped, because I needed to figure out the timing for my assembly,” says Legato. “I took that version to my basement, edited it together, and then used that as a template for the actual shoot. We used After Effects to do temporary composites of the actor’s video face on top of the original footage to, again, check body motions.”

On the day of the HD shoot, Legato lined up the actor’s faces using the same super-imposition method he used on the previz day. He taped the actors at 1080p/23.98fps, carefully matching eyelines and locking off the camera, keeping the actors’ heads still while they spoke their lines.

Legato also created a handful of insert shots, featuring PacificCare brochures being handled by actors. For those inserts, he filmed the brochures being handled by actors on 35mm film, and then color-timed those shots at TCS, allowing him to match grain and color in order to make the inserts combine seamlessly with source footage.

Eventually, after Legato cut the spots based on performance and timing, he created separate sequences for each spot that would later be combined — one background sequence from the source footage and two sequences of the actors’ facial footage, shot on HDCAM. All three sequences were played together to line up, starting from the first frame, playing in synch. Those plates were then shipped off to Image Metrics to track facial motion of the actors’ faces onto CG Fred and Ethel facial masks.

Legato particularly enjoys the “handcrafted” feel of the job. “You should be able to rewrite and change images as you go along as the director,” he says. “If you are talented and know what you are doing, you can do that, and should. Rather than being steadfast like the visual effects industry, for instance, which tries to hold you to things as you go along, or gives you a price penalty for changes, I can make the change myself if I think it will help. Working with high-powered directors like [Scorsese and Cameron] has inspired me to create this kind of work method.”