Optimizing Encoding Performance with Apple Final Cut Pro, Part 2

Figure 1. System preferences. Qmaster is on the bottom in the Other category.

Apple originally created Qmaster to enable Compressor and other rendering-intensive applications to use other Macs on a network cluster to share encoding tasks. That was back in the bad old days, when computers typically had single or dual processors and encoded very slowly. In this multiple-computer role, Qmaster could be challenging to set up, fine-tune, and maintain, because it involved separate computers on a LAN and sending huge files over that network to distribute the encoding.

Related Links

Optimizing Encoding Performance with Apple Final Cut Pro, Part 1 We've had a fun time so far with this column, digging inside an Apple Mac Pro to install a Blu-ray recorder and benchmarking Snow Leopard... Apple Snow Leopard for Video Producers, Part 2 In this issue, I share some benchmark tests with a Mac Pro and Mac Book Pro comparing Leopard (OS X 10.5) to Snow Leopard (OS X 10.6) in 32-bit and 64-bit modes... Matrox CompressHD Test Drive: Mac Psyched by the arrival of the outstanding liquid-cooled HP Z800 (Read about the Z800's liquid-cooling system), I has scheduled a review of Matrox''s hot, new H.264 co-processor board, CompressHD...

If you have a multiple-processor computer, you can also use Qmaster to help spread the encoding load over those multiple processors, which can dramatically boost encoding performance. In this role, Qmaster is much easier to set up and maintain than in its multiple-computer role. If you have a two-, four-, or eight-core computer and frequently render multiple files with Compressor, you really need to try Qmaster. In this article, I'll walk you through setting up and using this program.

To set up and start Qmaster, open your Mac's System Preferences window. On the bottom, in the Other category, click Apple Qmaster (Figure 1).

The Apple Qmaster configuration window opens. If Qmaster is running, click Stop Sharing on the lower right to stop the sharing so you can configure the program. On the top, choose the "QuickCluster with services" radio button. Note that this is critical, as neither of the other options will allow you to access the cluster from within Compressor. You know you've made a mistake if the "Identify this QuickCluster as:" text-entry box remains grayed.

Figure 2. The basic Qmaster options.

In the Services window, choose both Shared and Managed checkboxes for the applications you want to distribute over the multiple cores on your computer. I've selected Compressor only because I don't use Shake. Then click Options for the selected service button.

This is where you choose the number of instances available for Qmaster. There are two factors to consider: the number of actual cores on your computer and the amount of RAM. By default, Qmaster will select one instance for each processor on your computer. I've read some articles that recommend using one instance for each two cores on your computer, which will leave plenty of cores around for editing or checking email. Another recommended choosing one fewer instance than you have cores on your computer, or seven instances on an eight-core computer.

Figure 3. Choose how many cores to use during rendering with Compressor.

I would choose the default value and modify this, if necessary, based upon your experience. If you want to continue editing while rendering and the system becomes unresponsive, lower the number of cores. If you're rendering over a weekend, jack the number of cores to the max and go home.

For RAM, here's a tech note from Apple: "Be sure your computer has an adequate amount of RAM before configuring it to support running multiple services. A minimum of 1GB of RAM per service is recommended."

So, if you've got 4GB of RAM on an eight-core system, don't go higher than four instances. Or buy more RAM—you'll be glad that you did.

Figure 4. Advanced Qmaster options.

After you've chosen the number of cores to use, name your QuickCluster and deselect the "Include unmanaged services from other computers" checkbox unless you intend to create a multiple-computer cluster or access a cluster from another computer. If you don't deselect this and you have other computers running Qmaster on the network, their clusters will show up in Compressor, which could get confusing.

For example, on my network (right now), I have four Macs, all running Qmaster, though I don't share encoding between them. If the "Include unmanaged services from other computers" is enabled, when I try to choose a Cluster in Compressor, the clusters from these other computers appear. This is confusing, and if I choose the wrong cluster, Qmaster might try to spread the encoding over multiple computers, which isn't my intention. So, deselect this checkbox unless you intend to share encoding over multiple computers.

Figure 5. Make sure you assign the encoding task to the new cluster, not This Computer.

On top of the Qmaster configuration window, click the "Advanced" button to open those options. If you're not running Qmaster over multiple computers, deselect "Allow discovery via Bonjour." If desired, click the "Show Apple Qmaster service status in the menu bar" checkbox so you can see when Qmaster is working. You can leave the other options at their defaults and click Start Sharing to get the party started.

Figure 6. Your new cluster will get its own category in Batch Monitor.

Once Qmaster is running, you access it from Compressor—once you click Submit the first time to kick off encoding. Before clicking Submit again, make sure you select the cluster that you created rather than "This Computer."

If you're used to checking your work in Batch Monitor, you'll note that the Cluster gets its own category on the left hand side. Click that and the encoding jobs that you've submitted appear.

Figure 7. CPU use in the 96-percent range via Qmaster.

When you check Activity Monitor, for most encoding tasks, you'll see significantly higher CPU use. Though the red Compressor/ TranscoderX with (Not Responding) looks scary, it's probably appearing because the process is too busy to respond, not because it's about to crash. Here I'm encoding multiple files to H.264, and you can see in Figure 7 that the system is using all but 3.32 percent of available CPU, up from about 30 percent when encoding without Qmaster.

Table 1. Performance with and without Qmaster on a 2.93GHz, eight-core Mac.

How much will this extra CPU use reduce your encoding times? This depends upon the codec. Apple's H.264 codec uses multiple cores without Qmaster, though Qmaster makes it much more efficient for both single and multiple file encodes, as you can see in Table 1.

With other codecs, such as On2 Technologies' relentlessly single-threaded VP6 codec, accessed via Flix Exporter, Qmaster can't apply more than one CPU core to the task, so it doesn't affect single-file encoding times. As you can see in Table 1, however, if you're encoding more than one file, the time savings can be significant.

If you've tried sending an encoding job from Final Cut Pro 7 to Compressor via the new Share option, you probably noticed that Qmaster crashes the encoding operation in Compressor. I asked Apple about this when reviewing Final Cut Pro 7, and the Apple reps explained that on a single computer, Qmaster works by opening up multiple instances of the encoding program and assigning cores to each instance.

However, Final Cut Pro—which actually performs a portion of the encoding when you choose the Share command—doesn't support multiple instances on the same computer, and if you attempt to load multiple instances, it crashes the encoding. For this reason, you can't use Qmaster with the Share option on a single computer, though you can with a multiple-computer cluster. You can still access Qmaster on a Final Cut Pro 7 enabled system by rendering to a QuickTime reference movie, loading that file into Compressor, and then following the procedure outlined above.