King Kong2/15/2012 2:30 AM Eastern
Like the Lord of the Rings trilogy that preceded it through Weta Digital's production pipeline, King Kong is chock full of digital filmmaking innovations too numerous to list in their entirety here. Among them is a potential paradigm shift, whereby the film's visual effects pipeline and the digital intermediate/color grading pipeline were permanently bonded together under a single roof. Indeed, amid all the kudos Weta is getting these days for such things as CG hair and water, digital doubles, and motion-capture advances, the creation of a permanent digital intermediate department at Weta itself may be the project's most important development.
Indeed, at least at the high end of digital filmmaking, similar workflow approaches are already well in development across the industry landscape. Just prior to King Kong's debut in December, for instance, Deluxe Laboratories and Industrial Light and Magic (ILM) announced an agreement to open an Efilm Digital Laboratory capable of performing visual effects and DI color grading services, among other digital imaging services, in early 2006 at Lucasfilm's new Letterman Digital Arts Center (LDAC) in San Francisco. At the time of the announcement, officials from both companies noted a specific goal to achieve a more integrated workflow between visual effects and digital intermediate services.
As Weta ramped up for Kong, it built a huge DI department around a Discreet Lustre platform. That department included several colorists working on imagery from the start of principal photography on set — through dailies, previews, and the final DI — all reporting to veteran colorist Peter Doyle, who was officially credited as the film's color supervisor.
As Doyle explains it, this approach is new for major feature film production in that the film's digital color grading process was formally encapsulated inside a permanent department that collaborated closely with other departments throughout all stages of production and postproduction. In that sense, he says, color grading essentially mimicked the template of how visual effects procedures have evolved on major feature films in recent years.
“‘Color supervisor’ is essentially a new position, akin to the visual effects supervisor in that, like him, I'm basically working constantly with the DP and director to create the color brief for the entire film, and then working with a number of colorists and the visual effects people to get through a large volume of work to ensure the final product adheres to that color brief,” Doyle explains. “So my overall contribution is to keep everyone on track through all stages — making sure that all recorded versions, from online to video deliveries to the HD master to the final filmout and the D Cinema version, all are consistent. Those different versions represent many different color spaces, and so this job of color supervisor is designed to coordinate and ensure they are all working in concert to produce the best material in the best formats.”
Central to this mission was the decision to link the DI and visual effects departments in a single location at the main Weta facility in Wellington, New Zealand. Since King Kong's visual effects load (about 2,400 shots) exceeded any individual Lord of the Rings film, and, in fact, entered Star Wars territory, it was crucial that the production be structured this way, Doyle says.
“We had to have a DI facility internally in order to allow the integration factor to happen on a daily basis, meaning that all live-action plates and visual effects material could get creative direction and account for all costs with the final, overall color in mind,” he says. “Weta couldn't afford to take an ad hoc approach to doing an early color grade and then a final grade later where the effects were concerned. They needed a comprehensive pipeline. And so, on a daily basis, Weta was able to ensure that effects were graded in the correct color space and that everyone knew what that would be. This way, the final grade was about continuity, and visual effects were graded as if they were a seamless part of the dramatic piece, just like the camera negative. To perform that final grade at the visual effects facility is, in fact, a different model from how I, and most people, have done this in the past.”
Part of this integration included the availability of a Lustre workstation on set, permitting filmmakers to mull over the equivalent of what Doyle calls “colored previz” during the shooting phase. This gave visual effects supervisor Joe Letteri, DP Andrew Lesnie, and director Peter Jackson advance opportunities to make strategic color decisions about evolving visual effects shots in synergy with live-action plates.
“Scans were culled, viewed, and colored on set frequently, and that let Joe Letteri design what the background plates might need to be like in terms of color,” Doyle explains. “He and Andrew Lesnie were able to get a jump on grading the evolving plates in relationship to the dramatic live-action plates around them. This helped develop the visual effects more efficiently because it gave Joe a good indication of where to build visual effects, especially in terms of time of day cues and color palette. In a more linear pipeline, that process would normally be achieved by handing over a timed clip — an actual piece of the actual film print. But, with our compressed time schedule and the complexity and scale of this project generally, this approach gave Joe a big head start on getting visual effects where they needed to be in terms of timing.”
This methodology greatly pleased Lesnie. He says the move enabled him, among other things, to be around the visual effects' evolution. “Naturally, I found it difficult to avoid having an opinion,” he says.
“The point of having a Lustre workstation at the studio during the shoot was to give greater access for myself and Peter Jackson to the visual effects work,” says Lesnie. “This meant that visual effects conceptual work that kicked off during the shoot could have the benefit of a grade as it evolved. The bonus was being able to get [supervising colorist] David Cole to come on set and see what was going on. I could explain the direction I'm taking [on a particular scene] in the interest of clarifying our approach. There have always been issues about integrating visual effects with live action. If the look development of a sequence is happening concurrently with the development of visual effects shots within that sequence, it stands to reason that the two will be more compatible — especially if the lead compositors get to see where the grade is taking the shot.”
Lesnie emphasizes that this method was used only to make sure everyone was more efficiently on the same page regarding the evolution of color for the visual effects shots and the movie in general. Letteri still insisted that all effects shots arrive with a neutral grade to Weta's DI facility for final color correction, but, by that time, the direction was firmly set and understood by all parties involved.
Weta's DI facility included two Lustre grading systems (expanded to three this past October), Arriscan film scanners and Arrilaser recorders, and a staff of 15 people, including Cole serving as supervising digital colorist for Weta and lead colorists Billy Wychgel and Melissa Kangleon, who handled the final DI in collaboration with Doyle.
Doyle adds that Kong's synergy between visual effects and the DI was both necessary and beneficial to both departments.
“More than Lord of the Rings, Kong had many things that were complete visual effects constructs — so much of the lighting is really happening within the visual effects world,” he explains. “Color grading is a very useful tool for sitting down and handing over a sequence, and showing everyone that, ‘This is where it is sitting at the moment, so where do you think the grade should go from here?’ I can show them where it might evolve to, and then Joe [Letteri] can tweak a composite while knowing where the final grade on that shot is going to go. That is a very useful thing, and a new thing for major visual effects films.
“I don't know if this would be a trend for all films, but certainly, for films where a major component is visual effects and there is an overall intent to stylize and grade the film, the line between what is a final effect and what is a grade becomes blurred. This is important, since a color grade can completely break the intent of a visual effect if not done right. A densely composited visual effects sequence, and we have many of them in this film, is a very precious thing. You can only grade it so much or you double up on the work the effects facility has already done. It also means the facility can supply mattes and other things we might need, rather than having to rebuild a window in the color correction package. So to cross that line and get more shot components is a useful thing, and I think it gave us a better-looking result.”
While Warner Home Video recently released its newly restored DVD of the original 1933 version of King Kong to coincide with the theatrical release of Peter Jackson's theatrical remake, the new restoration project has been going on for about six years. That's when the studio began its search for enough source elements to produce a state-of-the-art restoration in the first place, according to Ned Price, VP of mastering for Warner Bros.'s Motion Picture Imaging division.
Price points out the original camera negative disappeared long ago, so a 1993 restoration was created from a 1942 release print and elements from a 1937 print used to provide missing censor cuts. Those elements, however, were in relatively poor condition, according to Price. He says, “[There were] notches for printing, which created film bumps [picture instability at grading changes], missing footage, [and] tape and cement splices for repair in picture and track, with frames deleted at edit points and frames deleted at the end of reels. The 1937 print also contained heavy vertical scratches from projection.”
The current restoration project, therefore, took the form of a puzzle assembly, as the studio scoured the globe for better elements that could fit together seamlessly and accurately.
“We were aware of the nitrate element held by [New York's Museum of Modern Art] — a 1942 print of the movie that everyone thought was a fine grain, which Turner Entertainment used as the backbone of their 1993 restoration,” says Price. “But it wasn't actually a fine grain, as it turned out, but rather a 1942 print on [Kodak] 1302 release stock. That stock was a bit finer than typical release print stock  or lavender  print stock from that era, so we were able to use some of it. But this element also has missing frames and dupe sections. It has those cement splices, notches, tape repairs, and severe flash frames at various cuts. Scenes originally censored and deleted from the film were part of that restoration, but you could see, judging from the print, how poor the condition of the original camera negative was, even by 1942. So finding material was probably the hardest part of this job.”
The studio's search led to six nitrate picture and five soundtrack pre-print elements of the film, plus a dupe negative of deleted sections that were found in the United Kingdom, where the movie was censored differently than in the United States. The U.S. version's censor cuts differed from the British version in terms of content, and the deletions made to the British dupe were retained over the years in a separate can.
“That let us cover for the poor condition of the scenes censored out of the U.S. version,” Price says. “Using this approach, we were able to manufacture complete shots. We would find a cut with a missing five frames, or whatever, and we'd then have to fill in the difference from other sources. A key scene in which the men on Skull Island fall off a log, for instance, always missed the end of the sequence, showing the impact of their fall at the end — strange things like that. But we were able to put them back together. In fact, our final restoration has an additional three minutes of material than [the originally restored version].”
The disparate elements were eventually collected and sent to Richard Dayton and Eric Aijala at YCM Laboratories, Burbank, Calif. They methodically went through all image and sound elements and built what the restoration team called “the road map” for the project.
“They had to go through every piece, shot by shot, cleaning and repairing them,” Price explains. “The picture elements were then sent to [Warner Bros. Motion Picture Imaging], where [colorist] Ray Grabowski completed a second electronic evaluation of all that material, causing us to make some adjustments to the road map. At the end of the evaluation, we had an EDL prepared for our 4K scan. A similar process was used for the soundtrack, which was transferred and restored [by Jim Young at Chase Productions, Burbank] from elements we took from unilateral, variable area tracks on the original British dupe negative, a 35mm composite nitrate print, and a 16mm print. After the 4K scan, it all went back to Ray, and he performed a comprehensive color grade.”
Grabowski explains that the big challenge he faced, working with a Pandora Pogle Megadef color correction system and 4K MTI software for scratch and dirt removal, was to build an even grade of the black-and-white piece that paid homage to the original look of the film, while, at the same time, fixing serious problems that cropped up over the years.
“I took the 4K scanned files from the [Grass Valley] Spirit 4K, and performed the grade using 2K proxies of those files so I could work in realtime,” Grabowski explains. “We went shot-by-shot, often frame-by-frame, for the missing shots we were restoring. That was common on almost every cut. Sometimes it was a full frame, sometimes a frame and part of another frame, and sometimes just half a frame, so it was a very difficult process. While I did that, my colleagues were getting rid of dirt and scratches using the MTI software.
“What was really complex was how many layers of grading we often had to apply. It got pretty thick at times. As problems mounted, we would add layers, meaning windows of areas to lighten or darken within a frame as I went along — sometimes up to six layers deep. On black-and-white film, that's amazing. But it was necessary, largely due to printing defects over the years.”
This situation amplified the key challenge facing any film restoration project — how to subtly make enough changes to improve the quality of the viewing experience as it was originally intended by filmmakers without overdoing it.
“There were lots of printing defects, and we had to deal with them, but we didn't want to artificially alter the film,” says Grabowski. “We didn't want to introduce modern artifacts to a film that was originally pretty beat up. We just wanted to figure out our changes and leave it alone otherwise — getting as close as we could to the original negative.”
Price adds that the restoration team therefore had to consider how special effects sequences in 1933 were shot and processed, and then strategically sorted out what subtleties to eliminate and which ones to leave alone.
“We decided, for example, to retain the [visible] props used for animation, though these would all be deleted in a contemporary film,” he adds. “Usually, you can tell what artifacts are inherent in the original photography, and what problems were created by subsequence printing and decomposition. We did try to eliminate the density flicker that cropped up due to the way they lacquered the original camera negative in that era. They did that back then to try and protect the negative from scratches, but as the lacquer decomposed over time, more dirt would get on it. If we had the original negative, we could have physically stripped the lacquer down — it's a scary process, but we could do it. Without the original negative, it was built into the fine grain, and we had to work it out electronically.”
There, Grabowski explains, the dirt-and-scratch fixing process proved crucial.
“Much of that work was done manually, but the MTI software allowed us to have a slight automated path to deal with the overall problem, and then we could get into it more manually, when we had to,” Grabowski says. “If film jumped in a frame, MTI would take care of some of it, and then I would finesse the rest. I could move physically in a frame using my color correction software, but if an upper third of the frame was warped because someone put splices over a sprocket, then we could get rid of that warp as long as there was no movement.”