Do you have a shelf full of hard drives that you've used to archive video projects? Have you built your own computer or thought about filling a storage appliance with a bunch of drives?
If so, you've probably struggled with the question of what hard drive brand and model to buy. You may have also questioned whether there is really any difference between consumer drives and premium priced "enterprise" drives.
It's easy to research price and performance, but reliability is tougher to measure. The challenge is the lack of unbiased data on on the lifespan of different drives. Vendors throw around terms like "enterprise class" or "server grade", but without large scale testing across vendors, it's difficult to see how different manufacturers and models measure up.
Fortunately, Backblaze, a cloud storage provider that offers unlimited personal storage for $5/month, has just published a blog post with some great information about which drives last the longest. This follows up on another post that looked at whether "enterprise" drives were more reliable than their consumer counterparts (the short answer, not in Backblaze's experience).
The thing that makes Backblaze uniquely capable of answering these questions is that it's one of the few companies with the storage scale (over 25,000 drives in its storage pods and counting) and culture of openness to provide meaningful data.
Since the service is based on commodity hard drives enclosed in its custom designed storage pods, Backblaze is using the same drives that you and I can pick up online. In fact, it has a very heterogeneous mix of drive capacities, brands, and models in its pods. So how do drives from different manufacturers stand up? The best indicator is the chart below that shows the annual survival rate by drive brand. Hitachi drives are a clear standout in this regard.
The chart below, with the annual failure rate for different drives tells a similar story.
Hitachi drives have a low annual failure rate, while the smaller capacity Seagate drives have high annual failure rates. It looks like Seagate's large capacity drives fare better, but Backblaze notes that it's keeping an eye on them because they are relatively new and Seagate drives have tended to perform well early on, with increasing failure rates over time.
Their blog offers a full breakdown of the different models from each manufacturer, capacity per drive, the number of drives, their age, and the annual failure rate. Although Backblaze uses some Toshiba and Samsung drives, it doesn't have enough of them to provide good statistical results.
If their buying decision were based solely on reliability, the choice would be easy. In their words:
If the price were right, we would be buying nothing but Hitachi drives. They have been rock solid, and have had a remarkably low failure rate.
However, they must buy drives based on a combination of price / performance in order to keep costs of the storage service low. To this end, their current favourites are the 4TB Seagate Desktop HDD.15 (ST4000DM000) and the Western Digital 3TB Red (WD30EFRX) for their new pods. The drives they will avoid when replacing units in their older pods are the Western Digital 3TB Green drives and Seagate 2TB LP drives.
Of course just because Backblaze buys certain drives, it doesn't mean that you should do the same. After all, filling a small NAS with devices or storing bare drives on a shelf is a very different environment than storage pods which run at high temperatures, have a significant amount of vibration, and run 24/7 for years.
For video applications price is going to be an important factor, so I took a look at prices of the broadly available 3TB drives that Backblaze mentioned (some are older units that are hard to find online). I've made the simplifying assumption that read / write performance is similar across the drives.
Which ones to pick for your video business? That depends on the target application.
Based on the data from Backblaze and the pricing data from NewEgg, I would probably lean towards buying the Hitachi drives for a smaller NAS or Thunderbolt RAID device. The low failure rate might be worth the extra $200-300 across a 4-6 drive array.
Why spend the extra money if you can easily replace drives in your RAID? My rationale is that RAID 5 (which is commonly used and is the default for Promise's Pegasus Thunderbolt storage solution), can only tolerate one disk failure at a time. If two drives go down at the same time, you lose everything.
I admit that the prospect of two drives failing at exactly the same time is small. The case that I would worry about is having a second drive fail while I am waiting for a replacement drive (you can still use the RAID with a missing disk, but the longer you do this, the greater the risk of failure), or some kind of hardware failure that corrupts data while rebuilding the NAS.
There is also the time and frustration associated with replacing a disk. Backblaze has employees who maintain the pods and replace drives on a routine business. The labour cost of doing this is part of the business model. In the case of most small businesses, if something fails you are going to spend time calling customer support, reading manuals, sourcing a new drive, and waiting for several hours while your NAS rebuilds itself. In this case, it may be worth buying the most reliable drives you can.
For the boot drive of a desktop system, I would probably shy away from all of the drives that Backblaze has used and opt for an SSD or one of hybrid alternatives like Apple's Fusion drives. If push came to shove, I would probably pick Western Digital over Seagate in this case, but it's hard to extrapolate Backblaze's results to this entirely different class of drives. Of course, I would make sure everything is backed up to a NAS that I can rely on. Based on the conversations I've had with ScreenLight customers, it seems like most video professionals store project media on external storage anyway, so the internal drive isn't as important.
For archiving bare hard drives on a shelf, I would probably opt for one of the Western Digital drives over the Seagate drives. Western Digital drives offer a relatively low failure rate at a nominal premium to the Seagate drives, which seemed to have a relatively high failure rate across the board. Whether this reliability carries over to storing drives on the shelf is uncertain, as there are plenty of reports of idle drives failing quickly if they aren't spun up regularly. I don't think I'd pay $50+ extra for the Hitachi drives in this case. I think this money would be better spent moving to LTO.
If I were keeping copies of data on bare hard drives, I would definitely use a master and clone configuration along the lines of what Walter Biscardi has outlined. This minimizes the risk associated with drives failing. I would also want to schedule regular times to check hard drives, spin them up, and replace as necessary.
For long-term archiving and off-site storage, the better solution of course may be LTO, but realistically, that's a big jump for many.
What do you think? Would you interpret the results differently or make a different purchase decision?
