cpn connect

Disks, Data and Demand: Developments in Media Storage

A few years ago we'd have been forgiven for assuming that by 2017, all storage would be solid state. 12/04/2017 4:00 PM Eastern

A few years ago we'd have been forgiven for assuming that by 2017, all storage would be solid state. After all, at that point, solid-state storage seemed new and exciting, and even if the future didn't use the sort of flash media we have in USB keys and laptops, it seemed likely that traditional spinning metal hard disks would follow floppy disks into oblivion in fairly short order. Of course, that hasn't turned out to be the case, and there's reason to believe it probably won't happen for a while longer.

Perhaps some of the expectations were guided by the feeling that solid-state storage was a new idea, though it emphatically wasn't. Anyone who'd used a Commodore Amiga PC during the 1990s would remember the chunky PCMCIA card (the MC meaning "memory card"), an almost credit card-sized device that could host all sorts of peripherals but had been intended mainly for storage. It's related to the CompactFlash cards we still use—they can fit a PCMCIA slot with a simple wiring adapter.

But even PCMCIA wasn't the first incarnation of solid-state storage. It's too involved a history to go through in full, but the idea of using electronics rather than spinning magnetic disks to store data goes back almost to the dawn of computing as an industrial tool. Let's consider solid state, non-volatile storage, which stores data without using mechanical parts (thus solid state) and without requiring a consistent power supply to retain data (thus non-volatile). The concept really dates from the 1960s or 1970s as a commercial product, often using EAROM (electronically alterable read-only memory), which was eventually dumped because it had a very limited rewrite count. Fantastically expensive battery-backed CMOS RAM emerged in the mid-'70s at capacities of a couple of megabytes, became popular in the early '80s and remained until the PCMCIA cards of the '90s.

Early PCMCIA could manage capacities up to a few tens of megabytes, and persists to this day in the form of Panasonic's much-upgraded and far more capable P2 card system for video devices. PCMCIA eventually gave way to the NAND flash we use today, initially at a few hundred megabytes and now a few hundred gigabytes per card. This sort of enormous storage expansion isn't unusual: at each stage of solid-state storage development, capacity and speed increased greatly, and predictions of the eventual death of spinning metal disks were common. Spinning metal, however, is still here.

The reason is fairly straightforward. Much as solid-state storage has grown in capability, so have traditional disks. Techniques such as perpendicular recording, giant magnetoresistance and shingled magnetic recording (all of which are fascinating, but beyond the scope of this article) have served to massively increase capacity, and if we pack more data onto a disk that's spinning at a constant rate, then the data is going past the read heads more quickly, so speed improves too. One of the ironies of innovation is that technologies are inevitably retired at the point at which they're most fully developed, and the traditional hard disk is now an extremely well developed technology.

The gap has been closing recently, of course, and traditional disk progress has slowed. As recently as 2012, SSDs were ten or more times the cost of a hard disk in terms of price per gigabyte. This chasm of price-performance ratio narrows to about four times by 2016, and on average less than three times at the moment (data here from Trendforce and Objective Analysis). To some extent, this measurement depends on the specifics involved: choose the most expensive, highly engineered enterprise-class hard disks, compare to the lowest-cost SSD, and the gap can probably be closed entirely. It depends on what you're trying to achieve, but there are two things to bear in mind: first, the cost per gigabyte of hard disks has reduced by about one-third since 2012, whereas SSDs have plummeted by almost six times. It's not clear whether that sort of dropoff in SSD costs can continue, or whether fundamental advances in hard disks will change things.

The upshot is that it's unclear, to within a few years, when SSDs will start to outcompete hard disks in terms of sheer space. Most of the numbers suggest that it won't happen before 2019, and possibly not at all, at least in the foreseeable future. One factor that's certainly true is that there simply aren't enough semiconductor foundries on the planet to entirely replace hard disks at this point. According to KDB Daewoo Securities, 2016 saw a demand of more than 120,000 million gigabytes (120 exabytes, 120 EB) of flash storage, with an excess manufactured of about 6 EB. That sounds like a lot, until we consider that more than 600 EB of hard disk capacity was produced in the same year. Flash isn't going to replace hard disks in the immediate term. Save a truly unprecedented expansion of extremely expensive microchip factories, we just can't make it fast enough.

As such, the choices are going to continue to be driven by application. Most of the flash built doesn't go into big business storage or even modular disks; it's soldered permanently into tablets, phones, USB keys and MP3 players—devices that must be physically robust, compact and low power. Solid-state storage also does well in applications such as web servers that need to access huge numbers of individually small files very quickly, because there's no delay involved in moving mechanical parts to different areas of a disk.

By comparison, the workload presented by film and TV work is fairly low-key. Yes, we like the speed of flash, and we might like its robustness and low power consumption when we're recording on camera. In postproduction, we can hypothesize a workflow in which a disk might need to retrieve lots of individual clips from different files, perhaps simultaneously during a compositing operation. In the main, though, on most workstations, post workflows involve long, streaming, sequential reads of a single area of the disk, and require a lot of space for the minimum money. Yes, SSDs are nice in this application, but if anything keeps flash storage out of video workstations, it won't be the technology. It'll be the fact that the disadvantages of spinning metal are the factors we often care least about, and the upsides are the characteristics we really want. It'll happen, but not tomorrow.  


Download the December 2017 issue of Digital Video magazine













Want to read more stories like this?
Get our Free Newsletter Here!

No Recent Articles