Imagine going to your favorite restaurant and watching social posts from your friends literally jump off the menu with recommendations for what to order, or getting a history lesson in urban growth as you ascend a building in which the elevator's walls come to life with immersive video, or putting on a headset to experience life from someone else's perspective. Digital experiences like these aren't taking place in some far-off, imaginary future. Many are happening right now thanks to emerging technologies such as virtual reality (VR), augmented reality (AR), mixed reality (MR) and voice interaction, which are reshaping the way we engage with the world.
Gartner predicts that by 2020, 30 percent of web browsing will be done without a screen, and by the end of this year, 10 million homes will have a room-based voice device like Amazon Echo or Google Home.
Digital experiences are moving beyond the screen. In the very near future we'll interact differently with both work and leisure activities in that smartphone, laptop, desktop and TV screens won't be the sole conduits for digital interactions. New technologies are emerging to create deeper connections between our physical and digital worlds.
How much longer will this setup be the standard for postproduction and design?
"I think every surface has the potential to be a digital surface," says Mark Asher, director of corporate strategy at Adobe (@Adobe). "Kiosks, immersive rooms, interactive walls and desktops, even our smartphones can be flipped into an immersive window that creates a 180° or 360° experience. All of this is becoming available around the world, and the world is being transformed into our own personalized digital canvas."
The Reality of Moving Beyond the Screen
While today we may be screen-focused, we'll soon see more experiences that aren't confined to those screens. AR, VR and voice platforms are boundless—and the technology gives the space you enter the potential to become a more intuitive, efficient and empathetic canvas.
VR/360° footage can still be edited on a laptop/desktop computer, but is that the most efficient way?
Advances in graphics processing and software integrations, along with the shrinking size and cost of hardware, are bringing immersive experiences to life today. "The ubiquity of personal and IoT devices containing low-cost sensors such as microphones, depth-perceiving cameras and GPS, alongside improvements in AI, are unlocking new possibilities for haptic touch, voice interaction, computer vision and gesture recognition, as well as contextual, situational and emotional awareness," says Abhay Parasnis, chief technology officer and executive vice president at Adobe.
Asher agrees: "We finally have the computing and the storage power necessary to deliver these experiences at a level of performance that is acceptable to us as human beings," he says. "We're able to suspend the disbelief that these experiences are 'artificial' because we are now able to deliver frame rates and the number of polygons necessary to make these virtual worlds look like truly immersive worlds."
Will your next editing suite look like this?
Taking Off the Rose-Colored Headset
Innovations in technologies are driving real-time immersive experiences, but there's still some progress to be made before we achieve the level of integration Parasnis forecasts. There are important challenges companies must overcome on the road to widespread consumer and enterprise adoption.
The reality is that this is an evolutionary process, and the hardware we have today has plenty of room to evolve.
"The main issue I have with immersive experiences is that I have to pull out my phone and look around, and the headsets are still clunky at the moment," says Stefano Corazza, senior director of engineering at Adobe.
Aside from physically burdensome hardware, software options are fragmented, and many require users to have experience with coding. While these developments offer a lot of potential from a creative standpoint, the tools remain difficult for creatives to work with.
Adobe's forthcoming Clover technology is an editing interface for VR videos in which editors actually use a VR headset to edit in real time.
As hardware improves, though, and immersive experiences become more commonplace, Asher is optimistic that our interactions with technology will improve. "I feel like we are on the cusp of finally bringing the 'humanity' back to computing," he says. "That means I'm now going to be freed to interact with digital experiences with my hands, or voice or a nod—using my body and space like I do every day with other human beings—instead of this sort of artificial concept of having to click a button and roll a mouse around a desktop to perform every task."
One issue to resolve before we achieve this ambition is the difficulty of editing a 3D video with 2D tools. Adobe previewed its upcoming Clover VR at the Adobe MAX conference last year to show how VR editing is evolving and becoming simpler to accomplish—and now Clover is headed into the latest release of Premiere Pro. Clover is an editing interface for VR videos in which editors actually use a VR headset to edit in real time.
Object tracking and projection systems in AR imbue the world around us with additional information.
Start Your New Reality
New beyond-the-screen technologies are evolving to democratize creativity and affect how we work. "We want a much broader audience to be able to create more AR and VR experiences than they are able to create today," Corazza says. "We want to allow people who are not necessarily coders, and not necessarily technical, to tell their story, and we want to allow those stories to be created on any digital device—particularly something as simple as a mobile device—and shared on any canvas."
Tell the Story of the Future
With almost 1 billion phones being AR-enabled in the next year and VR headsets evolving rapidly, immersive media is poised to become the next disruptive platform. Widespread adoption of immersive technologies by both businesses and consumers will make them standard tools sooner than we think. Silka Miesnieks, head of Adobe Design Lab, believes easy-to-use, immersive design tools will empower innovation and help the industry move forward.
Technology alone won't lead to immersive customer experiences, but telling an engaging story, enhanced and made real by technology, will. "It's always about telling a story," Corazza says. "Mixed reality is just another medium where you can tell a story that is more immersive, more personal and more contextual."
Companies and individuals who learn to combine compelling content with immersive technology will distinguish themselves from competitors and create a sustainable business that will help redefine experiences amid the transformation of the Experience Era.
Read more about the future of immersive experiences in Adobe's Beyond the Screen collection (theblog.adobe.com/collections/beyond-the-screen/).