Safe Digital Photo Storage: Hard Drives, NAS, and the Drobo

drobo digital photo storage
Matthew Gore | Light And Matter

For digital photographers, data backup is critical. In fact, attention to the storage of our photos has never been more important. Fifteen years ago, if our black and white negatives were properly washed, we could slip them into archival sleeves, stash them away in a box, and be confident that they’d still be around if we needed them in 100 years (the outlook wasn’t quite so rosy for color). Digital storage, though, is a different story. Most of us archive images on our hard drives (I have terabytes of files… optical discs just aren’t an option), but hard drives are like any other mechanical devices: it’s not a question of whether they will fail, it’s a question of when. If  I had a hard drive crash, I would lose years worth of work  and incalculable residual income and trust from my clients, not to mention all of those memories of my friends and family. I can barely stand to think about it.

Luckily, backup these days is cheap and relatively simple, depending on your needs.

Cheap and Simple

If nothing else, you can buy an additional hard drive for your computer, attach it, and use a backup program to copy your important files onto it. An internal SATA hard drive only costs about $60 for 1TB (roughly 1000 Gigabytes), and even 2TB drives are under $100. External hard drives are usually considered safer for your valuable data, as they keep your drive away from the heat of your CPU and case, and heat is one of the major contributors to hard drive failure. But don’t worry: even 2TB external drives cost under $100.

external harddrive for digital photo storage
Matthew Gore | Light And Matter

Again, depending on your needs, the backup software can be expensive, or absolutely free. Cobian Backup, for example, is a free backup program that allows you to specify which directories you want to backup (ie, C:/photos/ ) and where you want them to be stored (D:/photo-backups, for example), and it will automatically copy any new files from the former drive to the latter on a specified schedule. Of course, there is some potential for data loss between scheduled backups, but depending on how much work you do each day, the risk is minimal. More expensive programs, such as Acronis True Image, will monitor your workspace drive continuously and copy new data to your second hard drive immediately, which minimizes risk of loss, but also takes a slight toll on your computer’s performance.

Better yet, if you have the space in your computer and a motherboard (or expansion card) that supports it, you can set up a RAID1 array inside your computer. To go this route, you’ll buy two identical hard drives and attach them to your motherboard’s RAID ports, and configure them as RAID1 (consult your manual). In a RAID1 array, your first hard drive is mirrored exactly on the second, so if either drive fails, you’ll still have the data safe on the other. If you’re interested in using more drives, there are additional RAID modes that may provide better performance as well, but that is beyond the scope of this article.

High Capacity, High Performance

Network attached storage (NAS) devices have increased dramatically in popularity and dropped substantially in price. The idea is simple: instead of installing a hard drive on your computer, you attach a separate storage device to your network, and simply use it as you would any other drive on your computer. Standard, wired networks these days typically use 1Gbit network adapters, making the transfer rates twice as fast as is available with USB 2.0, and even wireless (802.11 N) networks offer speeds of 600Mbit/sec.  (as opposed to the 480 of USB 2.0).

netgear NAS for digital photography storage
Matthew Gore | Light And Matter

More importantly, though, NAS devices can:

  • hold any number of additional drives to meet your storage needs.
  • be set up as RAID arrays to protect against data loss from drive failure. Your data on the NAS is always safe, so there is no need for scheduled backups.
  • be accessed from multiple computers at the same time, which is important if you use a laptop and a desktop computer, or want to share the storage with family or colleagues.
  • usually be configured for internet access, so you can access your files no matter where you are.

In fact, NAS devices can offer dozens of additional features, depending on which model you buy. There are hundreds of makes and models, but many of the most popular models come from brands that we already know and trust, like D-Link, Netgear, LaCie, and Iomega. At the low end of the price scale, the D-Link DNS-323 holds two drives and costs around $170, plus the cost of drives ($60 each for 1TB). Keep in mind that if you set up as RAID1 (and it’s a good idea with only 2 drives), you’ll get 1TB of storage space if you install two 1TB drives because the second drive mirrors the first. Similar two-drive models include the NETGEAR ReadyNAS Duo, the Patriot Valkyrie Dual Bay(least expensive) and the Synology DiskStation DS211j (highest rated).

If you think you’ll need more storage space or more sophisticated features, the options are practically limitless. NAS enclosures suitable for professional photographers and small businesses start at around $300, such as the Netgear ReadyNAS NV+. However, most professionals find that for a more full featured, reliable, and easy to use solution, slightly more expensive models are preferable. The Synology DiskStation DS410 ($533) and DiskStation 411j ($396), for example,  are two of the best selling and highest rated enclosures in this class, as is the QNAP TS-410 ($430).

Easy and Flexible

Though similar in many ways to NAS, the Drobo line of products offer plug-and-play ease of use, require no configuration, and are easily expandable, even while powered-on and in use. They use a data-redundancy system that offers protection similar to redundant RAID systems, but it doesn’t require drives of the same size. However, most Drobos are not network devices; they plug into a single computer. If you want to share your data on a network, the Drobo FS line is also available.

The Drobo’s primary advantages come in the “ease of use” category; when it comes to performance, their transfer speeds are generally a bit slower than well optimized NAS systems, though not so much as to have any practical bearing for most applications.

[NOTE: After continued research and several comments from readers, I’ve decided that I can’t recommend the Drobo. Despite the company’s claims, tales of data-loss abound. Whether they are the rule or the exception, I can’t say, but they are too frequent to be ignored.]

Off-Site Storage

As long as I’m discussing storage, I’d be remiss if I didn’t mention online backup. There are several major providers, including BackBlaze, Mozy, and Carbonite. The major advantage of services like these is that your data is safe even if your house is burgled and computers are stolen, or if your computer is lost in a fire.  These aren’t exactly inevitable events in the same way that hard drive failure is, but they do happen. Most of these services provide unlimited storage space, which is great for photographers who work with large amounts of data.

On the other hand, they have some major disadvantages. Of course, there is the issue of ongoing cost, but it’s not terrible compared to the price of storage hardware. More important is the speed of access. On an average day while I’m out shooting landscape work in the field, I’ll fill two to three 4GB CF cards. I have a pretty standard high speed cable internet connection at home which gives me great download speeds, but uploading files is still quite slow in comparison. I recently uploaded a 1GB video file to Vimeo, and the process took over 4 hours. The math is simple. If I were to try to upload my work on a daily basis, I’d have more work to upload before the previous day’s work was halfway uploaded. But of course, the real issue is the thousands of GBs of photos that I already have archived on my computer. And, of course, if a wedding client walks in and says that she wants to buy a DVD of the high resolution files from her wedding two years ago, then it’s going to take me hours just to download them.

If you have a business-class internet connection, work or live at a university with fast internet access, or otherwise have access to an extremely fast internet connection this may be more practical for you, especially if you’re just getting started in photography. For someone like myself it’s simply not an option as a primary archive method, though it might work as a secondary archive for my best work.

A Few Words About File Formats

A friend of mine recently directed me to an article by a well known internet photography pundit, which made some abhorrent claims… but the most outrageous is that everyone should archive their digital photos in JPG format. The reasoning was not just that jpgs are smaller, but that RAW files require special converters and will be impossible to open in the future.

Let me simply say that the gentleman is wrong. If you archive in jpg, you’re losing over 90% of your data  (8 bit vs 12-bit or more) without even having a hard drive fail. Digital SLRs are more popular today than 35mm film ever was; only a complete breakdown of the fabric of our society is likely to bring about the complete loss of programs that will convert popular raw files.

However, if you’re concerned about saving your pictures as a more accessible format, jpg is not the answer. TIFF is just as legible as JPG without special converters, it uses lossless compression, and supports 16-bit image depth. A better solution, though, is simply to “convert” (it’s more of a wrapper than a converter) your files into Adobe Digital Negative format (DNG), which was designed specifically for the purpose. You can read about DNG format on the Adobe website.

Editor-in-Chief
Subscribe
Notify of
guest

5 Comments
Inline Feedbacks
View all comments
Craig Volpe

Thanks for the info Matthew (and Bill!). At one point I was thinking about getting a Drobo. The way they market it, it sounds like all you need is a Drobo and your data will be safe from crashes and drive failure because of the redundancy. My brother, who’s also a photographer, got one a while ago and put all his data on it. I don’t think he was making backups but just using it as his primary data location. Then it got corrupted and because the Drobo uses a proprietary file system, he wasn’t able to access ANY of the data on the entire array! That almost makes it worse than not using a Drobo because with individual drives you would only lose the data on the drive that fails instead of losing the entire array if it gets corrupted or the Drobo hardware fails. Luckily he was able to recover most of the data off of random drives he was using before he got the Drobo, but after that happened I noticed a lot of users complaining about similar issues with their Drobo. I think now he’s using a 8 drive Drobo and making backups onto the 4 drive Drobo but at that point you’ve lost the “ease of use” advantage plus it’s kind of expensive for what it is.

The method I’m using is a couple drives in my Mac Pro in RAID 0, which gives me a nice boost in performance. Time Machine keeps a live backup of important data and every night SuperDuper! wakes my computer up for a few minutes to do an incremental backup onto a couple other drives inside the computer. Then every week I physically swap the backup drives with ones I keep in the trunk of my car. I think it’s a pretty good solution considering it’s relatively low cost, has hourly/nightly onsite backups, weekly offsite backups, versioning, decent boost to performance, and is all automated except for swapping the drives every week or two. Much cheaper and safer than keeping all your data on a Drobo! Also, I was slightly concerned about storing the HDs in my car but I haven’t had any problems after more than a year so far. I put them in little hard drive cases which go in a watertight pelican case. If there’s a catastrophe like a meteor destroys my neighborhood the HDs will either be far away with me in the car, or I’ll be dead and not care about what happens to my data.

Thanks for letting me know about Backblaze. Sounds like a great deal for only $60/year! Might incorporate that as well for an extra step of security. Do you know if you can access individual files in your online backup or is it like a massive backup image that you need to restore in it’s entirety to access?

Craig

Bill Minton

Hey Craig,

I’m glad you’ve found a solution that works for you. It sounds like you should be fairly well covered.

I played with Backblaze during the trial, but settled on CrashPlan. I needed cross platform compatibility, and when I tried it, Backblaze didn’t work under Linux. I also had some issues with the fact that you didn’t have enough control over what got backed up and what didn’t. This hit me particularly hard with regards to how I store my pictures. It was forcing me to backup gigabytes of data that I didn’t want or need backed up.

If you do settle on Backblaze, take advantage of the trial and test things out (including restores) as much as you can. I don’t know about accessing individual files online via Backblaze, but you can w/CrashPlan.

Bill Minton

Another excellent article. Backups are something that for whatever reason, I have a genuine interest in. As such, I’ve spent a lot of time testing different applications,

scenarios, etc. and of course, have a learned a few things along the way.

1 – RAID isn’t backup. While that’s common knowledge to people who’ve spent much time working with backups, archival processes, etc., to the beginner, it’s not immediately

apparent. RAID configurations are meant to improve performance, absorb drive failure, or both, but they aren’t backups. It’s better to simply view them as just another disk

that still needs backed up. Even if you have a RAID 5 (can tolerate one drive failing) or RAID 6 (two drives failing) setup, it won’t help you get a deleted or overwritten file

back.

2 – RAID configs are nice when they work, and software RAID (setup using Windows for example) is more “portable”, albeit slower than hardware RAID (where a dedicated card or

on-motherboard “card” is used). However, when they fail, things can get ugly. If a RAID card fails for example, it often times isn’t just an *exact* replacement of the

problematic card you have to acquire, but often times the exact same firmware version must be loaded to see the data on the drives. This can be problematic if things have

worked fine for a long time, and you are now searching eBay for a card that hasn’t been manufactured for several years.

3 – NAS devices are attractive from a simplicity perspective, but can suffer some of the same issues RAID configs do. If your NAS device dies (as opposed to one of the drives

in it), you may be back on eBay looking for that model, and an old firmware version to load on it to access your data again. Drobo recently made their forum private so only

Drobo owners could see user posts (complaints?). That alone should be a red flag. Synology doesn’t seem to have gone down that road. Don’t take this to mean NAS devices or

RAID configs are bad, but there’s a world of difference between the 400+ drive EMC VMax SAN here in my office and the $400 4 drive NAS device supporting a couple of RAID configs

sold at BestBuy.

4 – Not all online backups are created equal. For example, Mozy has suffered a lot of bad press due to failing users in their time of need. It’s been quoted that their restore

% is around 99.99%. Carbonite hasn’t faired much better. Backblaze seems to be doing ok, and hasn’t yet caved to canceling the ~$5/month all you can backup service that Mozy

did, however, it is Windows only which will leave some users out in the cold. Personally, I use CrashPlan for offsite backup. Not only are you able to backup to their cloud

(CrashPlan Central), but also to internal & external drives, other machines you own, and even the machines of friends. The latter of course gives you free off-site backup.

It’s also worth noting that CrashPlan is cross platform, working on Windows, Mac, and Linux computers. Additionally, you can look into ways of utilizing storage services like

Amazon S3 or Rackspace, but they are priced based on how much storage (and sometimes transfer) you use, making costs add up quickly.

5 – It’s not really apples and apples comparing uploading a video to Vimeo with compressed, deduplicated backups. CrashPlan for example breaks up your files into blocks and

compresses & deduplicates based on those blocks. So, while you had to upload the full 1G file to Vimeo, it might actually be noticably smaller when backing up to CrashPlan.

Depending on the data, it might not be, but the chance exists that it could be.

So with RAID configs potentially offering additional problems, offsite backups taking forever to upload at current connection speeds, and recordable DVDs & blu-ray discs not

able to store enough to be feasible, what options are there? Plenty, you just have to remember a few things…

a – Redundancy – If what you’re storing is important to you, have more than one copy. Get two same sized drives (from different manufacturers) and “sync” from one to the other

using SyncBack, FreeFileSync, Allway Sync, or something similar. This will provide a 2nd copy, but it *isn’t* a backup!

b – Backup – Use some form of a backup program, one that allows you to restore prior versions of files. Compression, deduplication, and offsite capabilities are paramount.

c – Look into parity or recovery records. RAR files for example let you build recovery data into RAR archives, so if an RAR archive gets slightly corrupted, you can run a

repair on it and often times lose absolutely nothing. Two additional options for “parity redundancy” are Quickpar and disParity. QuickPar allows you to build parity files,

allowing you to set the redundancy %. For example, if you build QuickPar parity files for 25% redundancy, you’ll be able to fully recreate up to 25% of your data… *any* 25%.

disParity is slightly different, but non-less impressive. With disParity, you allocate a single drive for parity data, and that single drive can be used to rebuild any single

drive it protects. The killer part? That single drive can protect an number of drives. At first, it seems like some form of black magic, but think of it similar to RAID 5,

where no matter how many drives you have, you lose the equivalent of one worth of storage because of the parity data. disParity simply writes all of the parity data to a single

drive instead of striping it across all of them. There are some caveats, disParity works best when used to protect mostly static data (like pictures, music, movies, etc.) that

doesn’t change often. I can explain more about how it works if you’re interested, it’s really fairly impressive.

For my personal content, I use two drives, hooked to two different computers, and sync the data between them using FreeFileSync. This is my first layer of protection for most

common content (music, backed up DVDs, etc.).

For pictures, I go a little further. The last thing I want to lose is the 18,000+ set of pictures I’ve taken and cataloged of my little ones. For those I:
– Store on a primary drive
– Sync (duplicate) to two additional drives (different manufacturers)
– Store originals (pre-edited) in RAR archives with recovery records
– Protect RARs with QuickPar using 25% recovery
– Backup RARs with CrashPlan to CrashPlan Central and a family members house ~3hrs away
– Upload originals to http://signup.fotki.com/?inv_l=tkwtrfwrqqgt (that’s a referral link), which allows me to re-download originals via FTP if necessary
– Protect with disParity (which also protects other content as well)

Sorry for the long comment, but like I said, backups are something I enjoy, so I’ve spent a fair amount of time working with them.

Bill Minton

LOL, I hadn’t paid attention to the length of the comment vs the post, again, sorry for that. :)

I knew the blog post was fairly high level, and I do think it’s a very solid post. What I was hoping to do was give some thoughts and ideas to that user who was a step beyond that level…one who had already ventured into NAS territory for instance, and hadn’t yet thought about what happened if/when it failed, etc.

I think you’re spot on though, users should use this information for ideas and do the research to determine what fits their needs & budget specifically.

Again, good post!

Bill