Reducing Noise in Apple Final Cut Pro
A funny thing happened on the way to this episode of Final Cut Pro Insider. As you may recall, the last episode detailed the results of an extensive visit to iTunes, where I downloaded and analyzed about 50 files, and presented strategies for producing podcasts for the increasingly diverse range of iDevices. This episode was going to cover device presets in Apple Compressor, Sorenson Media Squeeze, and Telestream Episode. I hope this doesn't cause any riots in the heartland, but I'm going to delay that awesomely compelling subject for two weeks in favor of one that has more immediate relevance to yours truly.
Here's the back story. A couple weeks ago, I was in Los Angeles at Streaming Media West, a tradeshow sponsored by StreamingMedia.com, where I'm also a contributing editor. Sorenson Media was introducing a new product, Squeeze Server, and had volunteered to bring a camera and camera operator so that I could interview several Sorenson executives and those of their product partners, Aspera and RightScale, for the StreamingMedia.com website. The interviews were at Squeeze Server's launch party at the Rock Sugar nightclub, and though the camera operator brought an on-camera light, lighting was inadequate. The inevitable result was noisy video.
Said video was delivered to me already compressed in H.264 format, albeit at a very high data rate. Unfortunately, this only emphasized the noise, which obviously degraded the quality of the video. Suddenly, it seemed like a good time to get smart about noise-reduction filters in Apple Final Cut Pro.
Final Cut Pro doesn't have any native noise-reduction filters, so I tried the Noise Removal filter in Compressor. The filter worked OK, reducing noise, but adding a bit of blur that I could have minimized via the Sharpen Edge filter. Noise reduction is a science, however, and it didn't feel like Apple had invested much science in this filter. So, I endeavored to find a higher-quality alternative.
I Googled "noise reduction and Compressor," and saw a bunch of video producers who seemed to have the same opinion about Compressor's Noise Reduction capabilities. Though many third-party products were mentioned, the one that garnered the most digital ink was the Neat Video noise reduction plug-in ($99.99). So I got a copy and gave it a try, with very impressive results. As you'll see, the product dramatically improved the compressed quality of my video and offers a tiered interface that's simple enough for beginners, but can definitely go deep for editors who like to tinker.
If you buy (or even try) the Neat Video filter, I recommend downloading the manual, since it will definitely help you get a better result more efficiently. You start by applying the filter in Final Cut Pro as normal, then choosing a frame in the Canvas for Noise Analysis. Ideally, this will be a frame with "flat, featureless areas that contain no visible details." In essence, if there are no details, any content in those areas must be noise, which the filter will attempt to automatically detect.
After choosing the frame, you click Options in the Filters tab in the viewer, which opens the plug-in configuration menu, shown in Figure 2. Then you click Auto Profile on the upper left, and the filter attempts to find a "flat, featureless area" like the box shown on my right shoulder in each frame. If the Quality setting in the middle right is higher than 60 percent, you'll probably achieve a very good result. My frame scored 67 percent, so I could have clicked Apply on the lower right and started rendering.
If the filter can't find sufficient areas, you have to manually attempt to create the profile. During this process, you choose a rectangle yourself in any of the four boxes, which represent composite view on the upper left, plus Y, Cr, and Cb valuesor, if you prefer, composite and R, G, B. If you've picked an area that's not sufficiently uniform, you'll get the error message shown in Figure 3.
Once you choose a suitable smaller box, you choose Auto Profile and gauge your progress by the aforementioned score. You can also use the noise profile equalizer graph on the right side of the interface to assess your results (Figure 4). If all the points in the equalizer don't have values, as the red line doesn't in Figure 4, you select another square and click the Manual Fine-Tune button on the lower left of the graph.
The goal of all these semi-automatic adjustments is to find a filter setting that optimizes quality. If you're not happy with the results, you can also go completely manual, using multiple controls.
First, you can choose a filter preset, like one from the list shown in Figure 5. You could also eschew the presets, and adjust noise reduction and sharpening values in the Filter Settings tab (which replaces the Device Noise Profile on the right of the interface when you click the Noise Filter Setting button on the top middle of the interface). Read the manual carefully if you decide to fly solo, however, since there's a lot of guidance like not to exceed 60 percent filtering in the Luminance channel because it "may lead to loss of fine details and unnaturally looking (over-smooth, plastic-like) results." Sounds just like the first round of results I got before reading the manual.
There's also an advanced filter control set, shown in Figure 6 (and there's the Noise Filter Settings button on top that opens the filter control area). This enables a high degree of customization, particularly since you can preview your results in four different channels. I particularly liked the granularity of the Sharpening controls, which let me sharpen just the Y values (luminance) and adjust settings for High, Mid, and Low noise levels.
Once you're done tinkering, click Apply to apply the settings and return to Final Cut Pro.
All the adjustments applied to this point are intraframe, applied individually to each frame. Back in the Filter tab, you select temporal filter settings, which affect multiple frames in sequence.
Briefly, the Temporal filter radius determines the number of frames used for temporal filtration, with 1 (for some reason), meaning that three frames are analyzed. You can increase the number of frames, which increases quality, but also increases processing time. I encoded at the default setting and at 5, and I determined that the extra wait was definitely worth it, as you'll see below.
The filter threshold determines what the filter "sees" as noise as opposed to true motion. At high values, the filter sees more of the frame-to-frame changes as noise, which it attempts to filter out, which can eliminate true motion from the video. At lower values, it sees more of the interframe changes as true motion, which it then leaves alone, which can increase the noise in the clip. I used the default setting in all my tests.
Adaptive filtration tells the filter to adapt to changing conditions in the clip, which seemed like a good idea, while Mix allows you to mix the filtered frames in with the original footage. I was concerned that the totally filtered clip would look unrealistic, so I tried a value of 80 for Mix.
Those are the controls, but the proof of the pudding is in the eating. In this regard, I present Figure 8 as exhibit A. As you can see, I have three encodes: on the upper left, the temporal filter radius set to the max (5) and the Mix to 80; on the upper right, temporal filter radius at 1, Mix at 80; and on the lower left, temporal filter radius at 1, Mix at 100. The unfiltered clip is shown on the lower right. The three filtered clips look very similar in the figure, but if you play them you'll see a marked difference in the background noise.
Exhibit B are the video files themselves, which you can click to view in the links below. Note that these 720p files are encoded at about 10Mbps to preserve detail, so will take a few moments to download.
For the record, these clips are 11 seconds long, and joining me is Eric Quanstrom, Sorenson COO. I encoded the files using Compressor's iPad/iPhone 4 preset, and the nonfiltered file took 23 seconds to process. Maxtemporal.m4v took 3:36 (min:sec) to process, filter100.m4v took 2:34, and filter80.m4v took 1:44. Check out the files and draw your own conclusions regarding how much better the filtered files look than the original.
You can also play quandrant.mp4 which shows all four files in the configuration shown in Figure 8, and is the file from which the screenshot was taken. You'll see a very significant difference between the background and foreground noise in all four clips, particularly the background noise in the Max Temporal/80 mix clip.
All of us shoot (or are shot) in low-light situations, which inevitably results in noise. While not a panacea, the Neat Video noise reduction plug-in does a great job improving the quality of your video footage, with an easy interface for novices, and advanced configuration options for more technical users. It's hard to imagine any video producer who couldn't benefit from owning this affordable, highly effective plug-in.