Anyone who has worked in an office, or had a garage, has likely accumulated a collection of items on a desk, or in the garage, and at some point they likely vowed to purge the things no longer needed. However, a popular reason for not getting rid of the excess is the belief that as soon as it is gone, a use will be found for it. With that in mind, imagine the anxiety level of someone who has to decide how to store a digital mountain of video and audio data while still having access to it at a moments notice.
Spectra Logic’s T-Finity Tape Archive
B ut there is good news for those who tasked with making those decisions. Archiving technology has advanced enough so that the need for archive managers to decide what to keep and what to delete has been progressively reduced, thereby raising the question can everything be saved?
“That’s honestly the trend,” says Molly Rector, Spectra Logic Corporation’s chief marketing director. “People are leaning more towards saving everything forever because you never know what will be useful five years from now,” she said.
Most claims involving video date archiving are subject to debate, but there is no disputing that the amount of data being produced worldwide has grown exponentially. A recent Gartner survey shows data growth is the biggest data center infrastructure challenge. A little more than a decade ago, there was about an exabyte of data (one exabyte equaled one quintillion (1,000,000,000,000,000,000) of data bytes. That number had increased ten times by 2010, and it has increased another ten times through 2011. “Now that the world has created more data in the last year than was created in the first several thousand years of man’s existence the question is what can we do with it,” Rector said.
That has created problems for those who need to keep video data, says Rector, who provides Saudi Arabia’s security service as an example. In Mecca, the Saudis use their surveillance systems to record everyone who enters and exits Mecca, but—lacking any incident requiring investigation—the Saudi would only save the resulting video for up to four weeks. “Because of the way technology has evolved, on platforms such as ours, the Saudis can now save almost everything for longer periods and go back and look at trends,” she says. That includes reviewing the highest volume of visitors, identifying potential terrorist suspects and deciding what all that means in terms of overall security.
In November, Spectra Logic introduced, what the company says is, the highest capacity data storage system in the world. That configuration—which was created by combining eight of its “T-Finity” robotic library units—is capable of maintaining an archive of 3.6 exabytes of data, and it was. Spectra’s T-Finity base configuration starts with 100 “linear tape open” (LTO) slots and scales up from there. It operates robotically to check the health of the data saved with 30 different metrics during the life of the tape. The tape undergoes carbide cleaning to ensure the integrity of the data as it is loaded. Five years ago, 40 terabytes of data storage was ample for many organizations, but that is now considered just enough for a small company. Big organizations, especially those storing high definition video, are now archiving well into the pedabytes (a pedabyte is ten times great than 100 terabytes). In addition, research by Enterprise Strategy Group (ESG) forecasts that the total worldwide cumulative digital archive capacity will grow to 300,000 petabytes by 2015.
ViewCast’s NiagaraPro 2
DISk VS. TAPE
An ongoing change-over from tape to hard disk began picking up momentum a few years ago once the capital investment for each solution became about equal in costs, and within the past two years storage technology began migrating from hard drives back to digital tape cartridges. “When you get to one hundred terabytes, tape becomes less expensive from a capital investment standpoint,” Rector said.
T hose with smaller archive needs will still have a higher component of hard drives. Hard drives are considered by many to be easier to coordinate with a tape system. In fact, worldwide external disk storage system-makers revenues reached year-over-year growth of 12.2 percent, totaling just over $5.6 billion, in the second quarter of 2011, according to International Data Corporation.
According to ESG, tape currently commands a 38 percent share of the overall digital archive volumes and is set to experience a six-fold increase in digital archive petabytes stored through to 2015.
“Migration of media is a manual and tedious process,” says Rector, who adds that archive managers must migrate the data on a hard drive every three years, while migration of digital tape content is only required every seven to ten years making broadcasters enthusiastic about tape. In addition, broadcasters are migrating content from single definition to high definition, a process that requires ten times the original archive capacity.
B ut in choosing a system, the cost to performance tradeoff—how much data can be stored and how long it takes to access it—is the main criteria, Rector says. Assessing the amount of time data has already been stored helps with this decision. “Being able to get my content back in 60 seconds is sufficient if I’m able to store ten times as much,” she says. “That’s often where the decision between hard disc and digital tape storage occurs.”
POWER & HEAT
Power usage cost is also a factor because power costs are 15 times lower for archiving 100 terabytes of data on tape compared to archiving that data on hard drives. Contributing to those costs is the need to cool the heat produced by storage units.
With costs in mind, the return on investment (ROI) in a storage system is a central focus in the configuration of ViewCast’s Media Platform (VMp), according to Bob Morley, ViewCast’s director of VMp sales. ViewCast has compiled its customer’s ROI data, and the “VMp is saving them a significant amount of money,” he says.
T he VMp software application sits on IBM Content Management server hardware to integrate all the functions associated with the video production process. That includes the VMp talking to its content management application interface to manage stored data files and calling on the storage solution it has been configured to use. “What is key to the VMp architecture is that the storage solution can be designed with the right amount of performance, redundancy and capacity to satisfy the client’s business need and budget,” Morley says.
In addition, Morley says clients are at different stages of dealing with their digital archives. Some customers are simply attempting to organize assets that are spread out across their operation so they can be searched and retrieved, while others have come to the realization that their archive is now a strategic part of their business and will be used to generate revenue, communicate a consistent brand or deliver content. “It is this valuable asset that commands near-line storage and can justify the associated expense,” he says.
Morley says broadcasters that are transferring video stored on tape to disc need ingestion strategies that are aimed at capturing the video in the highest feasible quality for the application. An encoder is a key component, Morley adds. Viewcast’s Niagara Pro II encoder runs on two of the company’s proprietary software applications, Niagara SCX and Simulstream, pre-figured and ready to use. Simulstream allows creation of added data output from the same sources in different formats as well as simultaneous streaming in multiple resolutions, formats and bit rates.
The resulting product is then encoded in the VMp repository. Another option is to capture the video in a workstation running Adobe Premiere or Final Cut Pro. The video can then be placed in a VMp Hot Folder. Retrieved video can be sent anywhere from a content delivery network, website or physical media.
For users with patience and time to wait a bit, Masstech Group Inc. will soon be releasing its next-generation INDIGO asset management all-in-one suite. The INDIGO will allow broadcasters to manage all their media from ingest to editing and play-out without multi-vendor integration issues, according to the company. The INDIGO will feature IBM System x3650 M3 servers and Pulsar content quality verification throughout the workflow. The Indigo is designed so that there will be no additional costs or reliance on third party hardware to integrate critical functions. It will be equipped with the Masstech-developed Transcode Engine, a full library of format blades and operate on both MS Windows and Apple OS. It will present the look of nonlinear editing and support dual head VGA displays.
The operative phrase in asset management use to be “use it or lose it,” but the latest archiving solutions have changed the situation so it is possible to “eat your cake and have it too.”