Accelerating Special Effects

BlueArc Network Attached Storage

BlueArc Network Attached Storage

With a dedicated workforce comprised of hundreds of artists,producers, and support staff working against a demanding productionschedule, Rhythm & Hues Studios is a racehorse. Complicating ourneed for speed, the movement and storage of huge graphics files fromrendering processes pushed the studio from slightly more than 2TB ofstorage a few years ago to 10TB today. We estimate that will rise to40TB to 50TB within two years. While our primary function is thecreation of art, our infrastructure is all based around our use ofstorage, which is our lifeblood.

To solve the data storage issues we've encountered as ourinfrastructure has grown to accommodate our business, it becamenecessary to embark on a sweeping storage infrastructure overhaul.

Rhythm & Hues invested in the latest in desktop computing— dual-processor AMD-based workstations running Linux andequipped with 80GB hard drives — which helped the artists workfaster, better, and more independently. Yet our network data load anduser count made it necessary to devote a large amount of resources tomanaging bandwidth, since we have more than 450 desktop workstationsand 500 network nodes.

Moreover, most of our roughly 250 artists access and work on thesame scene files. These files continually increased in size, whileinfrastructure bandwidth and storage stayed the same. This practicecaused network and server bottlenecks that resulted in delays as thepipes and attached devices became overloaded. The bottlenecks in turncaused expensive downtime. For a company that was already running athree-shift, 24/7 operation, this was unacceptable.

Workers staring at the walls cost us dearly. Since 90% to 95% of ourcosts are labor, any time we come to a halt it's a very painfulexperience, especially toward the end of projects when the artiststypically work seven days a week to make deadlines. So any time we canpurchase storage technology that allows us to do things faster, better,and with less human intervention, that's better for the entirebusiness.

As an IT team, we knew that failure to address these storage andnetworking needs would decrease productivity. To tackle thebottleneck/downtime challenge, Rhythm & Hues needed ahigh-performance storage solution that could also deliver improvedavailability so users could easily and seamlessly access data.Additionally, a storage solution with a high performance-to-price ratiowould significantly help the company's bottom line.

Although Rhythm & Hues has historically been a NAS-centricorganization, we decided to review both NAS and SAN architectures. Wefound that SAN environments met neither our scalability norprice/performance requirements.

After we decided to continue with a NAS-based architecture, theRhythm & Hues IT staff evaluated systems from vendors includingBlueArc, Network Appliance, Scale8, and Zambeel. By focusing on fourcriteria — performance, cost, bandwidth, and custom solutions— our goal was to determine what solution could deliver the bestavailability and performance, as well as support super-high-capacitynetwork connections (all at an attractive price).

We needed a vendor that understood our business needs. For example,we don't necessarily always need just an off-the-shelf piece ofequipment. We need some things specifically tailored for ourenvironment. In exploring the possibility of developing a tailoredsolution, we even ventured into building our own NAS appliance.

We worked closely with the BlueArc engineering team to install theirSiliconServer for on-site evaluation. This process highlightedBlueArc's ability to provide a solution that met the demands of Rhythm& Hues' operations.

In the end, innovation won out. Rhythm & Hues chose BlueArc'sSiliconServers, largely because of their innovative architecture thatplaces the functionality typically offered via software intoprogrammable hardware. That architecture gave us a sort of NAS onsteroids by delivering a big boost in speed and a marked accelerationof throughput. In fact, it tripled device performance.

Within just 12 hours of installing the 1.5TB BlueArc server, thestaff had already filled 1.2TB of it. People were very happy and thedowntime was gone. It allowed us to remove all our data bottlenecks,which meant my system administration team could go on to dealing withother things instead of day-to-day trivial aspects of databottlenecks.

We now have three BlueArc SiliconServers, which connect via GigabitEthernet to switches from Extreme Networks. In this configuration, ourBlueArc storage solution gives us better performance than our priorsystem, including:

• The size of file reads and writes have more than doubled from 45MBreads and 20MB writes per second to 110MB reads and 90MB writes persecond;

• A scene file requiring ASAP rendering took six hours before, butonly 15 minutes afterwards; and

• Backbone bandwidth more than tripled.

In addition to these improvements, we can now take advantage ofsignificant cost benefits, such as:

• The cost per megabyte of storage has been reduced by 50% to 5 centsper megabyte;

• We could add 1.5TB of storage across the three BlueArc boxes for$25,000, one-quarter the cost of buying a new $100,000 server to handlethe extra load;

• The minimal administration and maintenance required by the BlueArcservers means that Rhythm & Hues can avoid hiring the threefull-time workers needed to support a DAS or SAN system; and

• Staying with a NAS system let us avoid having to buy the additionalhardware to switch to a SAN scheme.

The bottom line is this: We've solved what were daunting databottleneck problems with a new system designed to work well into thefuture. That includes the likely use of emerging 10-Gigabit Ethernetnetworking technology. The performance of the BlueArc SiliconServer,combined with decreased administrative time and expense, has allowedRhythm & Hues to re-allocate resources previously dedicated tostorage support into other areas that directly affect the quality ofthe studio's digital effects and animation. That's exactly what wewanted: a solution that could free us to put more work into the art andless work into the infrastructure.

Mark A. Brown is vice president of technology at Rhythm &Hues Studios, based in Marina Del Rey, Calif. With a history dating tothe beginnings of CGI in 1987, the work of Rhythm & Hues Studioshas gained top industry recognition for creative vision and productionefficiency in computer animation and visual effects.

R&H won an Oscar for Best Visual Effects for Babe(1995), and the studio's extensive list of box office work includesmany other films such as Harry Potter and the Sorcerer's Stone, TheLord of the Rings, Doctor Dolittle 2, and X-Men 2.

For more information, please visit www.rhythm.com