In this age of surveillance and social media, we're all Hansel and Gretel, leaving digital breadcrumbs in our wake wherever we go. Instead of being eaten by birds, however, the breadcrumbs are gathered and stored. Both online and in person, we exist in an environment in which our every movement is tracked and monitored. This scrutiny exceeds the knowledge and control of ordinary citizens.
That's the conceit behind the Park Avenue Armory's new installation Hansel & Gretel, a collaboration of architects Jacques Herzog, Pierre de Meuron and Ai Weiwei, whose previous joint efforts include the unforgettable stadium for the Beijing Olympic Games in 2008, called the Bird's Nest. As well as being an esteemed architect, Weiwei is an artist and activist known for his politically charged exhibitions, most recently focusing on the global refugee crisis.
Hansel & Gretel examines the psychology of surveillance in public spaces. The installation is split into two parts, placing visitors in the position of the observed and the observer. Entering the Armory from a back entrance, visitors must first walk a long, dark hallway before entering the equally dark and cavernous Drill Hall, where they must navigate the disorienting terrain. Projections appear on the floor, which visitors eventually realize are images of themselves captured from above. Visitors are then transformed into active elements in the work, free to do as they please while their movement is tracked and monitored from this god's-eye perspective.
Installed above is a grid of 56 small computers with infrared cameras, each attached to a projector suspended on a truss system above the floor. To allow better object tracking in the dim illumination, the room is bathed in near infrared light invisible to the human eye. Each computer runs a real-time tracking algorithm that isolates, identifies and follows each person in the room, triggering a red rectangular outline and grid pattern to appear. A white light follows the path of each individual across the Drill Hall, creating a visual record of visitors' movements before vanishing into the darkness behind them. The hardware and software powering this interactive playout system was developed by iart (Basel, Switzerland).
Also traversing the darkened space above installation visitors are surveillance drones that periodically survey the room. The automated indoor drone system used in Hansel & Gretel was developed by PhotoFlight Aerial Media and Easy Aerial. While neither autonomous nor indoor drone flying are difficult tasks, combining the two operations with multiple drones simultaneously presented a challenge. A network of 32 sonar beacons was installed in the Drill Hall to provide real-time position data that allows the drones to quantify their place in the environment and in relation to the other drones.
In lieu of a safety net between the drones and the public, which would have impeded the functionality of the installation, a special Drone Safety Suspension System (or DS3), consisting of a pulley and tether system, is used to ensure the safety of visitors.
There is an oddly peaceful sensation to the dark, quiet space that is only occasionally interrupted by the whirring noise of a moving drone and the realization that every movement is being recorded.
"We move forward like [Hansel and Gretel] in the forest, and instead of leaving traces behind us to find a way back, we are observed and our way is being traced on the ground," Herzog said during an artist's talk on the show's opening night. "And we somehow proactively and also inevitably create that drawing."
It is not until they enter the second part of the installation that visitors come to realize the extent of the surveillance and transition into the role of the observer. After leaving the Drill Hall, visitors exit to the street and re-enter the Armory from its proper entrance on Park Avenue. There, they are confronted not only with live video streams from the Drill Hall—along with an actual peephole that lets them surreptitiously observe those still in there—but six-foot tall wall-mounted screens that scroll through the visitors' own faces captured via infrared camera moments before.
Hansel & Gretel uses a custom facial recognition system developed by Adam Harvey that automatically captures visitors' facial biometrics as they move through the installation. At all times, five surveillance cameras stream an HD (1920 x 1080) image at 10 fps to a central image processing server located on site. Every frame from every camera is run through a series of computer vision algorithms that detect faces and convert them into biometric face prints.
At iPads set up at long tables, visitors can use the facial recognition software on themselves to find images of themselves traversing the Drill Hall. (The images are even available for purchase from the gift shop for $5.) The iPads also contain a range of materials on the topics of drones and surveillance technology.
Despite some of the more ominous aspects of the project, a sense of playfulness permeates it too. The setup of the space in Drill Hall is an invitation to engage and see what sorts of patterns you can create with your body. There is a strange satisfaction in seeing your face pop up both on the walls of the elegant building and on the iPad's facial recognition menu after you submit an image of yourself.
"It's to do with vanity," explains Herzog, who adds that anyone who uses a mobile phone or social media is complicit in putting their personal information out there. "Everyone wants to see his or her portrait. So it's very psychological. I think it's a very important aspect of the experience. It has to do with many layers of your life, the way to perceive the world and to be perceived."
This is a sentiment that Weiwei—an artist who has famously been under state surveillance in his home country of China for his work and outspoken dissidence—surprisingly shares. "I don't think it's a warning," Weiwei said about the decision to explore surveillance and use drones in the piece. "It's a mirror to our daily lives. As a New Yorker or somebody in the city, you expose yourself to surveillance all the time, no matter where you go."
Mass surveillance is a given in contemporary life. Though art and architecture cannot by themselves change the world, they can offer a new perspective on it and perhaps get visitors to think about issues surrounding the right to privacy in a hyper-monitored environment.
"It's very interesting that in such a big crowd, in such a massive city, our characters can be captured by digital pixels. And the images are so touching because it can tell that each individual is different," said Weiwei. "But of course it can be used for a very different purpose. It can be used by police or secret police or to limit freedom."
This unique mix of both whimsy and foreboding permeates even the gift shop where, in addition to purchasing their own "surveillance selfies," visitors can peruse merchandise that includes copies of classic dystopian novels 1984 and Fahrenheit 451, a poncho that prevents thermal imaging, and glasses that elude facial recognition software.
Hansel & Gretel remains at the Park Avenue Armory through August 6.