Creating and Delivering Content in 2018: Technology Advances from AI to VR

Many recent developments revolve around streaming, which we could broadly define as sending video over networks that weren't ever really intended to carry video.
Author:
Publish date:

Talking about what's coming up always feels dangerously like creating a technology buyer's guide. Giving advice about the current situation is bad enough in an industry where equipment can go out of date in less than a year, but it's even worse when we're literally trying to predict the future. Still, just as a surfer needs to start paddling at the moment the water begins to rise in front of a wave, making the best out of tech requires us to start working on the problem before it hits the mainstream. The difference is that it's usually pretty clear to a surfer that there's a wave coming, and how big it is. That, and surfing doesn't involve a five-figure investment in boxes covered in blinking lights.

Streaming
Many recent developments revolve around streaming, which we could broadly define as sending video over networks that weren't ever really intended to carry video. In distribution, that's old news, and the swing from traditional terrestrial and satellite broadcasting toward the on-demand platforms has been accelerating for long enough now that nobody will be surprised if it keeps on going. In production, the situation is slightly different. The standards process required for high-end video over IP networks has started to come together recently, and it certainly seems inevitable that the sheer flexibility and commodity hardware pricing of the IT world will eventually make IP interconnection almost universal. Even so, it's hard to imagine it happening to any great extent over the next year. New installs might do it, but otherwise it's likely that few people will be ripping out SDI for Ethernet in 2018. Even if they do, the difference to anyone but the engineering staff will be firmly in the details.

Image placeholder title

Improving public internet infrastructure could begin to replace satellite links.
Photo by Al Powers

What's more widely applicable is the fact that cellphone networks, at least in the more developed parts of the world, are starting to become a practical means to get video back to base. 4G LTE is theoretically capable of 50 Mb/s upload, with the more advanced versions capable of up to 150, and more is planned. Real-world situations achieve more like 25 Mb/s upload, but that's still sufficient for reasonably respectable HD video, and companies such as JVC have been offering cameras with network connectivity over Wi-Fi and 4G LTE for some time. This functionality is mainly of interest to news crews, who are embracing cellular transmission to the detriment of satellite operators. Still, there are plenty of places on the planet where a cellphone won't work well enough to transmit video, and that's likely to be the case for years yet.

Mobile Devices
The increasing capability of mobile communications networks is driven by the huge amount of work that's going into cellphones and related technology. In context, it's no great surprise that portable technology—principally that relying on ARM processors—is receiving the lion's share of attention. In some ways that's actually an alarming development given the importance of workstations in postproduction; ARM processors, to date, lack the sheer punch of Intel's most powerful options. It's possible that the balance is shifting, with AMD's Ryzen and particularly the Ryzen Threadripper range of processors moving back into a zone of realistic competition, but it seems possible that the lull has allowed developers using ARM designs to steal a leap on the market.

Image placeholder title

NewTek's TriCaster supports its NDI protocol, making computer networks part of a studio install—but that's not really streaming as most people understand it. (Pictured, NewTek TriCaster at TwitchCon.)

Just a few weeks ago, the first 64-bit ARM server CPUs were shipped by Qualcomm under the Centriq name. The reason this is happening is pretty clear: cloud computing is growing at an amazing pace. If we want cloud services, the computing work has to be done somewhere, which means big data centers full of server racks. The cost of the servers is one thing; a significant ongoing cost is that of power, and the promise of ARM was always that it would deliver significantly better performance per watt (or do the same work for less electricity). To be fair, it hasn't always been clear that this would actually be the case, but there's keen interest in the new chips on the assumption that they really will do more work for fewer electrons.

Whether they're useful for workstations is another matter. The Centriq line is intended to do a lot of tasks at once, a capability that is more useful for servers than workstations. There are already problems writing well optimized applications for multi-threaded computers; while work is underway to make it easier to do so, the effort is still theoretical. If ARM CPUs are to displace Intel in markets other than servers, there's a lot more work to be done.

Image placeholder title

Big racks full of hard disks still need to exist, but companies can avoid having to physically own them.

Returning to ARM's primary domain, tablets and smartphones, that market is naturally a lot more mature. It's no longer any surprise that phones do quite advanced things that have nothing to do with telecommunications. With the iPhone X, Apple introduced Face ID 3D facial scanning technology as a way to secure the phone, which is a great achievement for a consumer product. Even more impressive, the information collected for the Face ID depth map can be used for live facial tracking. Apple uses it to drive the animation of characters called Animoji, though others have experimented with the tech and have been able to capture both the shape and color of a face and make it available to the 3D graphics application SideFX Houdini. Facial scanning has been done before using consumer equipment: the Kinect sensor accessory for the Xbox was used in the same way, and there are pretty visible limits on the resolution of the 3D model. Still, if it's not enough that our phones are now used for communication, navigation, photography and all the other things we do, look out for ever-broadening horizons in the world of 3D scanning.

Virtual Reality
Speaking of horizons, the VR world seems poised to act on the fact that VR products can't establish a foothold in the consumer market at a $500+ price point. Oculus, the Facebook company that is perhaps most responsible for the current surge in interest in VR, has announced a $200 headset, Oculus Go, to be released early next year. The company isn't entirely abandoning the high end; a commercial version of its Santa Cruz prototype, which has more and better head position tracking options, should appear at some point in the next 12 months, price currently unknown.

Image placeholder title

A streaming studio setup at the NAB Show.

A price around $200 seems more reasonable for establishing a sustainable consumer market. With a 2560 x 1440 resolution, the new device is also an example of very high density TFT-LCD displays. With phone displays beginning to exceed the optical resolution of the human eye, at least at normal viewing distances, there was very little demand for more resolution until smartphone-based VR headsets came along. Whether the headset market alone can provoke and sustain development of these very high resolution displays remains to be seen. It could be, or perhaps has been, something of a chicken-and-egg situation, with sharper displays required to create sales, but sales required to fund the required R&D.

Still, with its significantly beyond-HD resolution and reasonable price, the Oculus Go is promising. What's far less certain is whether VR will escape the niche of computer games. It'll be no disaster if it doesn't; games are a ready-made application for VR, which has always been primarily associated with simulated environments. Games need very little modification to work well as VR applications, and the experience is widely enjoyed.

Image placeholder title

For a complete transition to IP, a wide variety of common devices, such as this multiviewer, must be updated for IP compatibility.

Alternative VR content such as 360° live video and pre-rendered experiences has been demonstrated convincingly. The Raid, a tie-in to the FOX series 24, was demonstrated at the 2017 NAB Show. In this 360° experience, the viewer accompanies a military unit on a night operation, with firefights and gunplay galore. Despite several high-profile successes, the relative difficulty of producing 360° material and the almost completely untried nature of the art form have left it playing second fiddle to the world of interactive entertainment.

Artificial Intelligence
Finally, while it's difficult to point to any recent advance in artificial intelligence that might have propelled it to prominence, IBC this year was one of a few places where AI-based technologies were talked about (a lot) and shown (a little). The fundamentals of neural networks are a bit too much to go into here, but their key advantage is that they allow us to break certain rules, or at least bend them, by bringing in outside information. For instance, it's impossible to sharpen a poorly focused shot because the blur is a problem of missing information; the high frequency detail of the shot has been lost forever. An artificial intelligence can, in theory, solve this problem and clear up the image based on its knowledge of what the world should look like, just as a human retoucher could.

Image placeholder title

At the 2017 NAB Show, the immersive virtual reality experience Tree from Microdose VR was on display at StudioXperience.
Photo by Al Powers

Real-world examples tend to be more prosaic—the prototypical example is handwriting recognition—although AIs capable of solving problems have begun to emerge. Recently, a team of researchers from Disney showed an application capable of generating photorealistic clouds in a fraction of the time required by existing mathematics. Clouds are difficult. They're made of billions upon billions of water (or ice) droplets that may reflect and refract light in complex ways, bouncing and bending it many times before it exits the cloud. Current approaches involve approximating that diffusion, extending the number of refractions or reflections that occur until the cloud's appearance is acceptably realistic. It's very slow, so the new approach solves a problem for Disney.

While artificial intelligence and neural networks may be buzzwords, they are, unusually, buzzwords with real capability. Whether any of these developments is likely to trigger fundamental changes to working practices is another matter. The key capabilities in film and television work are subjective, artistic and skill-based, and all of these tools are really aimed at making it easier for skilled humans to apply their abilities. The technology most likely to make big changes is AI; since it represents such a huge computational load, it'll likely improve as computers and cloud networks improve. Given the inexorable improvement in computers, then, we may be welcoming our whirring, beeping robot overlords sooner rather than later. Here's hoping it doesn't take all the fun out of filmmaking. 

Image placeholder title

Download the January 2018 issue of Digital Video magazine

Related

More than 55 of the 2018 Fall Television and Streaming Series Rely on Blackmagic Design

More than 55 of the 2018 Fall Television and Streaming Series Rely on Blackmagic Design

Blackmagic Design today announced that the company’s production and post products were used to complete many of the fall 2018 season’s new and returning television shows and streaming series. More than 55 shows rely on Blackmagic Design’s digital film cameras; Fusion visual effects (VFX), compositing, 3D and motion graphics software; and DaVinci Resolve editing, color correction, VFX and digital audio software; as well as its switchers, routers, monitors and capture and playback devices.

Luxul_AMS-1208P

Jingle All the Way: Luxul's XMS-1208P Managed Gigabit Switch is Available for All CI Integrators this Holiday Season

Luxul, the leading innovator of IP networking solutions for AV integrators, today announced that it's giving CI integrators the gift of enterprise-level performance at affordable price points this holiday season. It doesn't matter if they've been naughty or nice, the company's XMS-1208P 12-port/18 PoE+ managed Gigabit switch — that delivers high-speed performance while enabling simple network expansion — is now available for all. And to ring in the new year, Luxul will begin shipping its AMS-1816P 18-port/16 PoE+ L2/L3 managed Gigabit switch in early January.

HoverCam_VV1

HoverCam Transforms Classrooms Fit for Future-Ready Students at Val Verde USD

HoverCam, an innovative technology leader in the digital education market, announced that Val Verde Unified School District (USD) in Parris, California, successfully installed HoverCam's Pilot digital teaching stations and CenterStage interactive flat panels (IFPs) in new STEM labs in the elementary schools, with plans for an eventual rollout to the district's middle and high schools. As part of the district's goal to create future-ready students, Val Verde USD chose HoverCam digital education solutions to address the rising challenge of updating classrooms and introducing curriculum with advanced systems that will prepare students for a rapidly changing career landscape.

ScreenShot2018-12-11at2.41.29PM.144822

The Sigma Holiday Sale Is Back!

Nothing makes gift giving as enjoyable as special prices on award-winning Sigma lenses and accessories; get the lens you’ve always craved or surprise a fellow shooter with the perfect present.

Riedel-Ruptly-Vans-3

Global News Agency Ruptly Relies on Riedel's MediorNet and Artist on Board New OB and DSNG Vehicles

Riedel's MediorNet real-time media network and Artist digital matrix intercom are providing the decentralized and redundant signal routing and communications backbone on board two all-new, state-of-the-art OB vehicles for Ruptly, a Berlin-based international news agency. Qvest Media, a world-leading system architect and integrator for the broadcast and media industries, designed and built the new broadcast van and DSNG vehicle.