Return of Weta's Effects Wizards

Sidebars
2001: The Fellowship of the Ring
2002: The Two Towers
Peter Jackson

New Installment Seals Trilogy's Place in Digital History

Photo: Pierre Vinet / ©2003 New Line Productions

Shortly after the early December world premiere of The Lord of
the Rings: The Return of the King in New Zealand, Joe Letteri sat
down in his office at Weta Digital in Wellington, New Zealand, and came
to the realization that for the first time in several years, Weta, the
digital pearl in Peter Jackson's down under filmmaking empire, was
not filled to capacity. In fact, the facility was actually
emptying out in early December for extended holidays before much of its
staff—some of whom had been working on the LOTR trilogy
for as long as seven years in one form or another—would regroup
in early 2004 to swing into preproduction on Jackson's next film, the
upcoming King Kong.

After conceding that production of the trilogy really was over,
Letteri, co-visual effects supervisor, along with Jim Rygiel, on The
Two Towers and The Return of the King, finally had time to
sit down and discuss the contributions of the Lord of the Rings
project to the advancement of the visual effects industry.

Weta artist works on one of hundreds of complex models incorporated into effects shots for The Return of the King.

"I really believe that filmmaking generally, and specifically visual
effects, are more mature because of this project," says Letteri. "Peter
Jackson wanted to create a visual effects facility for this project,
but as the three films progressed, you see that instead of building a
facility for Lord of the Rings, Weta has become a filmmaking hub
to bring crews in and out, on top of a core crew, to develop tools and
techniques, to R&D filmmaking approaches, and to combine
off-the-shelf standard tools with proprietary technology in order to
service any aspect of the filmmaking equation.

"Everyone in the visual effects industry is familiar with the tools
that are out there, but the ability to maintain a staff and an
infrastructure to serve all these different areas is much more
difficult, and only a few have done it. In essence, this facility has
become a production facility of its own, with its own preproduction,
production, and postproduction capabilities. Digital effects are next
door to a workshop for puppets, props, and miniatures, editorial is
here, the sound department is right around the corner, the stages are a
couple of blocks away."

Letteri also points out that LOTR, when viewed as a single
project, simultaneously contributed to several industry technical
advancements that normally take place piecemeal on a variety of
different projects.

"Subsurface scattering for the creature Gollum, artificial
intelligence for animating crowds, complex facial animation, muscle
systems for creatures, on set motion capture, and so on—we
weren't the only ones working on these things, but we were working on
all of them simultaneously over the last few years," he explains.
"That's a fairly significant development."

The Next Level

If the trilogy as a whole has made these kinds of contributions,
then The Return of the King represents more than the end of the
series—it represents a spectacular culmination of this
breakthrough effects process, a process which has already been honored
with two Academy Awards, with another one likely this year.

Letteri says the effects mission of the third film was to up the
ante considerably from the first two efforts.

Peter Jackson directs a scene from the trilogy. Photo: Pierre Vinet / ©2003 New Line Productions

"The big thing we had to do that was different from what we did for
the first two movies was to create a larger sense of scale and scope
for the big finish of the whole thing," Letteri explains. "The first
film sort of gave the audience a view of the locations and a feel for
this magical world and the strange creatures in it. The second film got
darker, but mainly foreshadowing things to come, and introducing the CG
character Gollum prominently. But the third film is designed to advance
the sense of scale in a big way as everything climaxes. Thus, while we
were able to create the big battle for the second film—the Battle
of Helm's Deep—mainly with a combination of location plates,
miniatures, and CG additions, we had to take a different approach for
this movie's big battle—the Battle of Pelennor Fields. This
battle features an army at least 20 times the size of the army seen in
the Helm's Deep battle. When Peter Jackson realized we could do 20,000
soldiers for the second film, he asked us to do 200,000 for the third
film, and so things grew leaps and bounds in terms of scope between the
two movies."

Letteri says Weta therefore had to create "far more than we could
possibly photograph. Yes, we had plates of these big, beautiful,
sweeping locations in New Zealand, but we had to change that very
landscape so extensively to stage the battle on top of it that it made
far more sense to build Pelennor Fields as an entirely new 3D
environment. With so many live characters, 3D characters, miniatures,
and other elements, this was the only way we could create sufficient 3D
moves, helicopter shots, camera drops, and so forth to give the battle
the sense of scale it deserved. This is a good example of how our
approach evolved over the years—we built the third movie's
battles in a far more sophisticated way than we built the earlier
battles. We didn't throw away what we learned earlier, but we built
upon it considerably."

Gollum Evolves

While such issues rose in their complexity for the third movie,
however, other parts of the job got easier, or at least more creative,
according to Letteri. In the case of the all-CG creature Gollum, for
instance, the focus switched from creature-building techniques to
integration.

In fact, by the time The Return of the King entered the
effects' stage, the animation process for creating the odd creature was
essentially unchanged from The Two Towers. For both films, actor
Andy Serkis performed the part both on set for benefit of the other
actors, and on a motion capture stage. The data from those sessions
then made its way into sophisticated 3D versions of the character,
augmented by a complex system of automating over 300 different muscles
within the character. That motion-capture data, massaged by veteran
animators, accounts for the character's lifelike movements and
look.

Gollum evolves, or rather, devolves, in the third film, however, so
Weta's artists spent much of their time doing more with him than in the
last film, according to Letteri.

"We had the technique by the time we started work on the third film,
and we knew our tools better," he explains. "But in the third film, he
does things and goes through things that don't happen in the second
film. For instance, he becomes totally scarred and beaten down, and
that includes a physical transformation as the character unravels.
Because we had perfected our technique for creating the character,
during production of the third movie, we were able to spend most of our
time working on this transformation. What's cool about this is that we
could focus on creative aspects, not mechanical aspects. We spent more
time dealing with integration and environmental issues related to
Gollum than on building the creature himself. Once again, this was an
evolution from the first two movies to the third movie, and the movie
benefits greatly from it."

Indeed, the third movie marches so rapidly past the first and second
films in terms of its creature and environmental work, it's almost
impossible to describe. The Return of the King is a veritable
feast of Oliphaunts, reptilian Fell Beasts, the giant spider known as
Shelob, Orcs, and so much more. "We pushed our tools and our facility
about as far as we could possibly go on this one," Letteri says.

Even so, he fully expects Weta will have to go further in the
creature department, at least, for the upcoming King Kong, which
is already being prepped by Jackson's team. "But at least that's only
one movie," he concludes with a chuckle.

When Peter Jackson and his collaborators launched the leap of faith
that became The Lord of the Rings trilogy, they were more than
seven years away from the slick pipeline and state-of-the-art, Frank
Lloyd Wright-inspired facility that Weta Digtal would become. One of
the first innovations was Jackson's decision to shoot all three films
together during a marathon 15-month shoot between mid-1999 and 2000.
That shoot would win cinematographer Andrew Lesnie an Oscar for The
Fellowship of the Ring, an award he dedicated to chief lighting
technician Brian Bansgrove who passed away just as the first film was
released. "Brian was my main reason for winning," says Lesnie, citing
his collaboration with Bansgrove during an epic preproduction period,
where the team worked out the lighting scheme that would differentiate
each of the three films. For Fellowship, the approach was the
most fanciful, emphasizing faces and eye light, creating a glow of
innocence that would unravel as the trilogy moved into darker
times.

To read more on The Fellowship of the
Ring check out these Millimeter stories:
VisFX March 2002

Sidebar

2002: The Two Towers

While the lifelike Gollum was the poster boy for the accomplishments
at Weta Digital, the visual effects innovations that went into The
Two Towers spanned nearly every part of the production and post
process. As director Peter Jackson took Middle Earth to war, Weta gave
him the tools—including computer graphics with artificial
intelligence. With their proprietary Massive software and Grunt
rendering system, the effects team created armies of CG soldiers who
could react to their environment with a repertoire of patterned
responses. One of the most dramatic examples is a battle at Helm's
Deep, where 10,000 CG soldiers march upon the fortress, throw ladders
against it and scale the walls. Using motion capture of live actors,
the team amassed libraries of motion information and then "trained" CG
soldiers to be climbers, runners, or jumpers, able to react randomly
and differently to each other. This is just one example of a workflow
that would eventually reach beyond computer graphics to embrace
emerging technologies such as digital dailies, digital intermediate,
and digital mastering as well.

To read more on The Two Towers check
out these Millimeter stories:
In the Rings
AI Armies
Gamesmanship in Lord of the Rings, The Two
Towers

Sidebar

Photo: Pierre Vinet / ©2003 New Line Productions

Peter Jackson told Millimeter that his Homerian quest to make
the Lord of the Rings trilogy was "the most worthwhile project
that I would never, ever attempt again." During 15 months of principal
photography, Jackson flew all over New Zealand, supervising as many as
seven far-flung units, as they simultaneously captured the three films.
His three years of prep, working closely with production designer Grant
Major and producer Barrie Osborne paid off in the field. "One of
Barrie's first jobs was as an assistant on the Apocalypse Now
shoot in the Philippines. So if he could handle that, then the rest of
us can handle this, I guess," Jackson said. The post process would
prove even more groundbreaking as Weta Digital built a digital
infrastructure that literally grew in realtime to meet the challenges
of the project as they evolved and in turn influenced the storytelling.
"Digital technology made this film possible," Jackson states. "We
couldn't have done this trilogy 10 years ago. Technology brings a
different reality, which this story requires."

To read more on Peter Jackson check out
these Millimeter stories:
Interview with Peter Jackson