Using eye-tracking tech and machine learning, 'The Angry River' narrative changes based on where a viewer looks

What if viewers – not editors — determined the trajectory of a story? Filmmaker Armen Perian intends to find out with the release of his new short film, “The Angry River.”

Perian, along with tech partner Crossbeat New York, have created a film with the interactivity of a video game but without a game controller.

The idea came to Perian during a long editing session. “Someone said, ‘Man, I wish we could just edit this thing with our minds,’” he says. “It was totally off the cuff, but the idea stuck with me.”

His new short film, starring Jim Beaver ("Deadwood") and Brooke Smith ("Bates Motel") employs eye-tracking technology to determine what a viewer is watching – where the viewer’s eye lingers — and then edits itself into one of five possible storylines, each matched to the viewer’s interest.

While enamored with the idea of game theory, Perian "wanted to preserve a cinematic experience, even though you’re watching it on your computer."

Like any other film, a viewer watches the screen, but when the viewer pays attention to something on screen, that drives the action, and the movie changes based on what is being viewed.

A custom-built algorithm turns the five different perspectives into a story that looks and plays like a traditional movie.

A custom-built algorithm turns the five different perspectives into a story that looks and plays like a traditional movie.

"You watch it like any other film. But watching something is still making decisions, and what the viewer decides to pay attention to ultimately drives the action. It changes the movie they see."

Read more: The Next Big Step For AI? Understanding Video

From Armen Perian's short film 'The Angry River'

From Armen Perian's short film 'The Angry River'

Crossbeat New York developed the eye-gaze detection and machine learning that power the story’s algorithm. Perian directed the film, which was shot in Oregon over four days. Alex Hall edited the film into five distinct narrative tracks. A custom-built algorithm turns the five different perspectives into a story that looks and plays like a traditional movie.

"I resist the idea that this project puts me in the tech world. I'm still a filmmaker, a story person. I just wouldn't be able to tell this story without the tech."

Related

BlackBox-Radian

Black Box Radian Flex Software-Based Video Wall Platform Earns ISE Best of Show Award

Black Box, an industry-leading provider of pro AV connectivity and signal distribution systems, today announced that the company's Radian Flex software-based video wall platform won an ISE Best of Show Award from AV Technology Europe, a Future Publishing magazine. Of the more than 1,000 exhibitors at ISE 2019, only 15 — including Black Box with Radian Flex — earned this award, which recognizes achievements and innovations in product and service development.

Luxul_XMS-1208P

Luxul to Bring Reliability and High Performance to IBS 2019 With Latest Networking Solutions for the Smart Home

Luxul, the leading innovator of IP networking solutions for AV integrators, today announced its technology line-up for the NAHB International Builders' Show (IBS), taking place February 19-21 in Las Vegas. In the Legrand booth SU614, the company will exhibit a wide range of solutions for home builders that deliver reliable, high-performance networks in smart homes, including Gigabit switches; the next generation of wireless controller technology; indoor and outdoor wireless access points (APs); wired and wireless routers; intelligent network power distribution units (PDU); and the Wi-Fi Assurance Program exclusively for builders.

HoverCam_Pilot3

HoverCam's Pilot Series and ClassFusion Learning Platform Land Big Wins at TCEA 2019

HoverCam, a leader in innovative, interactive, and engaging education solutions, announced that the company's digital classroom solutions received three major honors from Tech & Learning at the recent Texas Computer Education Association (TCEA) 2019 Conference in San Antonio. At the show, HoverCam's Pilot 3 digital teaching station received the Award of Excellence in the District category, while the company's new Pilot X podium and ClassFusion software platform both earned Best of Show honors. The awards were presented on Feb. 6 at a special ceremony held at Casa Rio restaurant in San Antonio.

Veritone_CollegeFootballPlayoff

College Football Playoff Uses Veritone's AI and Digital Asset Management Solutions to Enable Richer Media Coverage

Veritone Inc. (Nasdaq: VERI), the creator of the world's first operating system for artificial intelligence, aiWARE™, today announced that the College Football Playoff (CFP) leveraged an expanded deployment of Veritone solutions and services for the 2019 CFP National Championship to give members of the media even faster and richer access to its content resources. In addition to the CFP's Digital Media Hub, a white-label digital asset management portal from Veritone, the CFP also utilized Veritone's Live Event Services and aiWARE artificial intelligence platform for the first time. With aiWARE, the CFP was able to apply artificial intelligence to video footage and still images of press conferences, interviews and other Media Day events to produce transcriptions, automated facial recognition of players and coaches, logo identification, and other tasks enabling expedited and enhanced coverage of the 2019 College Football Playoff National Championship. For the first time, the CFP was able to make Media Day press conference clips and images available for search and immediate download within minutes of delivery. Following the success of this deployment, CFP has renewed its agreement with Veritone through the 2020 championship game in New Orleans.