Optimizing Video for Streaming

If you work with moving pictures — video or film — youprobably spend a good chunk of your waking hours thinking about imagequality: how to achieve it and how to maintain it. So it may not beeasy to get excited about a medium in which the label“near-VHS-quality” actually has a positive connotation.

But like it or not, the streaming of video over networks is here tostay, another piece of the ever-expanding media pie that keeps us allfed. As broadband deployment grows to cover (eventually) the majorityof American homes and businesses, more and more viewers will bewatching their video streamed. And no one has a bigger stake than theproduction community in ensuring that the quality of those streams isas high as it can possibly be.

Obviously, the key factor driving streaming video’s popularityis convenience. “It’s ‘what I want, when I want,where I want,’” says Randy Thomas, director of marketing atTeranex, Inc. in Orlando, Florida, which develops compressionalgorithms and tools for maintaining the best possible image-qualitywithin available bandwidth. “It all comes down tovideo-on-demand.”

Although many have claimed that online movies is the ultimate killerapp for streaming, the reality at the moment is that the greatest --and most profitable – use of streaming is coming from the area ofeducation and corporate communication. With streaming, every school,institution, and corporation can have, in effect, their own personalbroadcasting network.

“Education,” says Matt Cupal, senior VP for strategy andbusiness development at Sorenson Media in Salt Lake City, Utah, thecompany that developed the video codec used in QuickTime streaming,“will benefit so much more from readily available video becausemuch of what we learn is from personal interaction with and observationof a teacher. You can't see the countenance of a HTML page.”

Likewise, such benefits can be realized in the workplace. “Inthe corporate market, where there is plentiful bandwidth on intranetsand virtual private networks, streaming is being used increasingly tocommunicate with company employees,” says Greg Lowitz, generalmanager of the webcasting solutions group at Pinnacle Systems inMountain View, California, whose broadband production and Web encodingsolutions include StreamGenie and StreamFactory. “Whether it'sthe CEO talking about recent financial performance, or a productmanager launching a new software release, streaming helps peoplecommunicate their message quickly, convincingly, and with impact,particularly in corporations with remote offices around theworld.”

Maintaining Standards

Once we accept the utility of streamed video in various fields,it’s realistic to recognize that untrained viewers probablywon’t be in the front lines demanding higher quality. Instead,technical and creative professionals will have to advocate for themaintenance of quality standards through to delivery, especially ifbottom-line business types seem ready to settle for less. Of course, tobe an effective advocate, you need to know what options are availableto make things better, and when to use them.

It would be convenient if there were a single step at whichvigilance is important, or a single process that magically makes allthe difference. But like any other production process, the preparationof video for streaming is a chain that is only as strong as its weakestlink. “Each part of the chain is crucial to the success of anystreaming event,” Lowitz observes.

“If you don't have the expertise to encode well,” Cupaladds, “you'll kill the content before it even gets off yourmachine. Delivery is also key. The Internet is a cold, dark place wheremany a packet can get lost. A great delivery network is key tominimizing this danger. And of course viewer-bandwidth has a tremendousimpact on final viewing quality. 300kbps means near-VHS quality; 28kbpsmeans postage-stamp. Finally, preparation of the video is important.It’s key to know how to shoot content for online consumption aswell as how to properly store it.”

Lowitz says the guiding principle in optimizing video for streamingis to maintain "contribution quality" video and audio all the way up tothe point of encoding. “Where possible,” he says,“use component video, either S-VIDEO, DV, or SDI. This providesthe cleanest video image without artifacts due to dot crawl. For audio,use balanced connections to keep noise and hum to a minimum. Make surelevels are optimized for the best signal-to-noise ratio. And use acompressor/limiter, if available, to keep audio levelsconsistent.”

If you are actually shooting your content with streaming in mind,rather than simply preparing existing footage, you’ve got evenmore variables to work with. Good practices begin with understandinghow the video compression algorithms (codecs) do their job.

“Compression basically works by finding high correlationbetween pixels that are adjacent in either space or in time,”Cupal says. “More similarity means more correlation means bettercompression.”

Asked for basic shooting tips, Cupal says to “use agood-quality camera, use a tripod, shoot lower motion shots, and usesolid colors rather than patterns in clothing and sets. Avoid slowpans, and avoid shooting outside with trees as background.”

Pre-processing

Once the video is captured, there is still much to do to make itwork better for streaming. “Depending on the target bit-rate,pre-processing of the audio and video image prior to encoding cansignificantly improve video quality,” Lowitz says. “Whilethe latest codecs from Real and Microsoft are already excellent,additional pre-processing can improve quality another 20 to 30percent.”

“You need to takes steps to ensure that you are not encodinginformation that is not required in the final output,” saysCurtis Palmer, CTO and co-founder of Sonic Foundry, the Madison,Wisconsin company whose media creation and editing tools include VegasVideo, Sound Forge, and ACID. “You need to start by capturing thebest-quality source video possible from a properly-calibrated system atfull resolution.”

If the content originated on film, Palmer continues, inversetelecine should be used to bring the video back to its true frame rate.He also suggests running an intelligent de-interlace algorithm, ifneeded. “The idea is to ensure you have a progressive scan sourcebefore encoding it in any streaming media format.”

“Inverse telecine is quite important for film content at anybit-rate,” Thomas concurs. “By removing the redundantfields inserted during 24-to-30 fps film-to-NTSC conversion, aguaranteed 20-percent bit-rate reduction may be achieved with no lossof picture quality. If encoding the 24fps signal requires 20-percentless bits, the bit-rate can be turned up by 20-percent, improving thepicture quality accordingly. The trick is to remove those redundantfields in the midst of bad edits, broken cadence, and orphan fields. Ittakes quite a bit of processing power to address all of the possiblepermutations of mixed 3:2 pull-down.”

The next step would typically be noise reduction. “Noisereduction helps at all bit-rates,” Thomas says, “but moreso at lower bit-rates where every bit counts. By removing informationthat is not important relative to the content, a significant number ofbits may be saved in the stream. Saving bits results in less of thevisible picture content being discarded during the quantizationprocess.”

Once you have the cleanest source possible, Palmer advises croppingthe source video to remove areas that do not need to be encoded. Asexamples, he points to both letterboxing and the overscan area thatwould not normally be visible on a standard television.“Streaming media can display 16:9 aspect ratios withoutletterboxing,” he says, “so why feed this to theencoder?”

The final steps he suggests are to “use a high-quality scalingalgorithm to reduce the frame size and adjust for non-square pixelsfrom the video source. Also, a high-quality frame rate converter may berequired for low bit-rate encodes. Note that with the proper tools,such as Vegas Video, many of these steps can be accomplished in asingle pass.”

For audio, Palmer says, the steps are analogous to video:“Capture the best source you can. Clean it up if needed. Adjustthe levels and use a peak limiter if needed. Use a high-quality samplerate converter if you need to reduce the sample rate.”

Choosing the Tools

Tools to optimize video before compression are available in manyforms. They are frequently built into encoders, including software suchas Media 100’s Media Cleaner Pro, the optional HiQ module forPinnacle’s StreamFactory, or standalone hardware such asTeranex’s StarStream Compression Station.

One factor influencing the appropriate type of tool to use iswhether the streaming will be on-demand or real-time. “With livecontent,” Cupal says, “you’re constrained to theconnection’s bandwidth at any given second, as well as theability of the video encoder’s CPU to keep up. Today’s CPUsmax out at roughly 320 x 240 at 30fps in real-time. This works well formost applications and can be VHS quality, but it's not the qualitypeople are getting from our high-end VOD stuff. With VOD you can playtricks like two-pass compression that do wonderful things for yourvideo quality.”

“For VOD, human-assisted multi-pass encoding enables thebalancing of bit-rate versus subjective quality,” Fritsch agrees.“Live Webcasting is more difficult. You don't have the luxury ofsophisticated pre-processing, and you don’t get a second chanceto encode the content.”

Live or on-demand, much of the optimization process is dictated bythe target bit-rate. “Know your audience,” Thomas says,“and the distribution bandwidth to them. This will determine howmuch and what kind of optimizing you will need to do.”

As for specific features, Thomas says the most important to look forare “intelligent inverse telecine, high-order down-samplingfilters rather than just field deletion, and de-interlacing that takesmotion into account. Automatic scene-change detection can also help byplacing the key frames in the optimal location for codingefficiency.”

Of course, the equipment by itself can’t do everything; italso takes the judgment of an experienced operator. “Even thoughtools have improved considerably in recent years,” Palmer says,“they still require that the user know how to use them to get thebest results. A little knowledge about preparing content for encodingto streaming media can go a long way. On the other hand, not knowinghow or when to use certain tools can lead to disastrousresults.”

Lowitz agrees. “With so much focus on the technology behindstreaming,” he says, “it's easy to forget that a productionis only as good as the talent, the production values, and the content.In particular, since Web video still has certain limitations in thedelivery of very high-quality images, even more attention is requiredon the production itself to optimize the limited bandwidth available,and to create a video stream that is both compelling and engaging forthe viewer. So the talent behind the scenes can rest assured that theirskills are now more important than ever.”

Phil DeLancie is a freelance writer based in Berkeley,CA.