When some people get a powerful new computer, they want to produce a new video, play games, or show it off to their friends. I just want to run as many tests as possible to analyze its performance, which I guess means that I”m well suited for this portion of my job description.
This navel-gazing aside, I have in my hands an Intel Nehalem-based Apple Mac Pro, specifically a 2.93GHz dual-processor, quad-core unit running Mac OS 10.5.7 with 12GB of RAM and an ATI Radeon HD 4870 driving a beautiful Apple LED Cinema Display. In this month”s first installment, I”ll compare performance of this computer with that of two older Macs running a variety of programs. Next issue, I”ll open up the case and delve into options such as a RAID hard-disk controller, multiple graphics boards, and different RAM configurations, assuming that I can convince someone to send me 24GB of RAM in 4GB dual in-line memory modules (DIMMS).
Briefly, in my review of the Mac Pro for millimeter, I described what the new Nehalem-based Intel Xeon CPUs brought to the table, which essentially is increased throughput between main memory and the CPU, and faster CPU performance. My hard-hearted editors frown on my using the same material twice (or, I should say, charging for it twice), so I won”t repeat the bulk of that information. I”ve looked at the Nehalem CPU a few times now, and you can find the most comprehensive description, albeit within the context of a Windows workstation, here.
I ran three sets of tests for this article, one of which is included in the Mac Pro review, which I”ll include for completeness. Let”s start with a brief review of those.
Table 1. Apple Final Cut Studio comparative testing.
Real-world Editing Trials
I tested three applications—Apple Final Cut Studio, Adobe Media Encoder, and Telestream Episode Engine—comparing the results to those from a one-year-old 2.8GHz eight-core Xeon system with 10GB of RAM kindly provided by Carrie Cannaday, a shooter/editor of broadcast, wedding, and other event work in southwest Virginia. Note that by virtue of processor speed (2.93GHz to 2.8GHz) the Nehalem-based computer should be about 5 percent faster than the older-style Xeon. I present the first set of results in Table 1.
The Final Cut Pro tests involved a 30-minute wedding ceremony shot by Cannaday in HDV with multiple camcorders. I rendered the ceremony to a QuickTime Reference Movie, input that file into Compressor, and produced an H.264 file for streaming and a DVD-compatible MPEG-2 file. The comparative times are shown in Table 1.
As you can see, the Nehalem-based system was only 22 percent faster than the older-style Mac Pro (17 percent after adjusting for CPU speed) when producing the QuickTime Reference Movie, but the numbers jumped significantly for the heavy-duty encoding, totaling 91 percent for producing both compressed formats. Note that total encoding time doesn”t equal the sum of the parts because I encoded both files simultaneously by enabling Apple Qmaster.
Table 2. Adobe Media Encoder comparative testing.
The second round of tests involved the Adobe Media Encoder and two of my own real-world projects. The first was a 90-minute ballet shot with two HDV camcorders mixed via Premiere Pro”s multicam feature and rendered to MPEG-2 for an SD DVD. In this test, the Nehalem-based Mac Pro was 79 percent faster than its older sibling.
The second and third tests involved a 10-minute single-camera DV shoot produced as an audition for America”s Got Talent. I rendered the file to H.264 and to MPEG-2. As you can see in Table 2, for this simpler project, the Mac Pro”s performance advantage was more modest.
Table 3. The results from Telestream Episode Engine tests.
The next tests involved Telestream Episode Engine, a streaming-media encoder with very efficient multiprocessor use. Here I ran two tests, the first encoding a single HD file to nine different streaming formats, a mix of VP6, H.264, and Windows Media output in different resolutions and data rates. The second involved encoding 16 1-minute SD source files to 14 output files in the same three streaming formats, plus MPEG-2.
In the first round of tests, the Nehalem-based system was 31 percent faster, while proving 77 percent faster in the second, which likely reflects the fact that encoding 16 SD files involves much more I/O than encoding a single HD file to multiple formats.
Table 4. Results for synthetic tests.
Synthetic NLE Tests
While I had Cannaday”s computer, I ran a series of multiple-format benchmark tests that I developed when testing Windows and Mac computers to determine the optimal RAM configuration for Adobe CS4. These tests are obviously synthetic because unlike the previous CS4 and Final Cut Pro tests, these are not real-world projects, but projects I compiled specifically for testing.
I ran two tests with all four formats shown in Table 4 on both computers: one a short project of 2 minutes or less, the other 10 minutes long. The short project involved multiple picture-in-picture effects, including an Adobe After Effects chroma key effect incorporated via Dynamic Link. My goal was to stress system memory and simulate the production of a heavily edited but short project such as a 60-second commercial.
The second round of tests involves 10 minutes of lightly edited source material, including color correction and a logo, but no picture-in-picture or Dynamic Link. This test was designed to assess pure throughput, in essence to see how Nehalem”s increased data bandwidth would perform in the typical event-type production such as concerts, ballets, and sporting events. In all tests, I timed how long it took to render the project to DVD compatible MPEG-2. Table 4 shows the results.
Back at you with the Apple Mac Pro. In our last segment, I looked at performance. Here, I”ll look at serviceability and the performance of Apple”s RAID option…
Apple was the first out of the gate with an Intel Nehalem-based workstation, in the form of the Mac Pro announced in early 2009…
Sporting a completely redesigned case and Intel”s new Nehalem processor, the HP Z800 knocks the socks off HP”s existing workstation line…
This month, I”ll review how the number of processors and the amount of RAM impact encoding performance, and then find other, non-CS4-related fish to fry…
On one level, these results are pretty impressive, with performance boosts of between 8 percent and 44 percent (3 percent to 39 percent after adjusting for clock speed), essentially realized because you decided to buy a Mac Pro in May 2009 and got the Nehalem-based unit, rather than buying the older-style Xeon in February. Who knew?
On another level, you have to wonder why the Nehalem-based Mac Pro yielded a 79 percent performance boost in my real-world HDV project, but only an 8 percent benefit in the long-form synthetic HDV test. My guess would be that the difference relates to the real-world test being a two-camera project and the synthetic test a single-camera project. Since Cannaday needed her computer back for real work (go figure), I was unable to test this theory.
Even more confusing was the fact that in all four formats, the Nehalem-based Mac proved more beneficial in the shorter project than the longer project, which was the reverse of what I found when running these tests in Windows. After testing in Windows, I had concluded that the features that most directly improve Nehalem”s performance are the integrated memory and Quick Path Interconnect architecture that accelerates data throughput to and from the CPU. That”s because in Windows, Nehalem accelerated performance in the long-form test—which involves lots of data, but relatively little processing—much more than the shorter project. Here, the reverse was true.
What conclusion to draw? At least as it relates to HDV, when synthetic test results collide with real-world test results, go with the latter. Whether short- or long-form, the results in all other formats clearly illustrate the Nehalem-based Mac”s improved performance.
Table 5. More streaming encoding tests.
Other Streaming Encoding Trials
This next series of benchmark tests compares the performance of the new Mac Pro with an older-style 3.2GHz Mac Pro running Mac OS X Leopard with 8GB of RAM. Unlike our first two sets of test, where the Nehalem-based box enjoyed a 5 percent speed advantage, here the Nehalem was about 9 percent behind the faster, but older-style computer.
There”s just one catch: I ran the tests on the older Mac Pro back in December 2008, and then reluctantly returned the computer to Apple. I obviously ran the tests on the new Mac Pro in May and June using the latest releases of the three programs in Table 5. During the interim period, all three programs in Table 5 had minor point releases (e.g. from 3.01 to 3.02), but no major new releases (e.g. from 3.0 to 4.0). That”s why I excluded both Sorenson Media Squeeze and Telestream Episode Pro, which released new versions in the interim that offered both faster performance and improved quality.
So while the results aren”t as perfectly apples-to-apples as I would normally like, the across-the-board performance increase tends to indicate that the Nehalem-based Mac is a rising tide that lifts all boats in the harbor. In particular, Compressor”s H.264 encoding performance with Qmaster enabled was almost scary fast.
What does this add up to? Between the real-world and synthetic tests, it”s clear that the new Mac Pros delivers substantially improved performance over previous Xeon generations. In particular, if you”re a producer of long-form, multiple-camera HD projects, you probably will yield very significant time savings when it comes to rendering and outputting your projects. In streaming-encoding-related chores, those encoding multiple files to multiple targets will yield the most benefit, but all should experience significantly improved performance, even over previous-generation 3.2GHz Mac Pros.