Storage Overview

Introduction

Digital photos are easy to make, duplicate and send out, they are also easy to lose. A single mistake or incident of equipment failure can wipe out an entire image collection. In order to preserve your photos for both the short and the long term, you will need to understand some basic principles of digital storage. These same principles are at work, whether you are storing a few hundred gigabytes of photos, or many terabytes.

We will start this lesson by outlining how to think about storage and backup, and how to design a storage system that is economical. In later lessons, we will take a look at specific storage technologies.

Storage is a process

I wish that I could tell you that digital storage is simply a matter of buying the right device and putting your files away. Unfortunately, there is no digital storage device that is anywhere near perfect. Hard drives can fail without warning. CDs and DVDs may become unreadable quickly if they are mishandled. Online storage services may go bankrupt and simply go away.

And even if the media that the digital files are stored on survives intact, you can run into other problems. The media may become obsolete, and it may be difficult to find a device to read it.

At minimum, good digital storage includes several components: a clean and neat way to store a primary copy of the files, a way to backup the files that can be used to restore them in the event of data loss, and periodic migration of the image files to new storage media at the appropriate time. I will look at each of these below.

Primary storage

The first step in making a digital storage system is creating a primary location for your files. You should know, without any doubt, where the main copies of your image files are. This primary copy of the files should be organized in a stable, orderly and scalable way. If you get this part right, then making a secure digital archive can be pretty easy to do. It will also help you to streamline your workflow. I will now explain these terms.

Primary - this is the main copy of your files. You need to know which is the primary copy of your images and which copies are backups. If you do not have this straight, everything else will be confused as well.

Stable - When you put your photos away into the primary archive, they should stay where you put them. One of the main reasons people lose files, is that they start moving stuff around. In the Lightroom Workflow course, we will see how you can use that programme to organize your photos without having to constantly move stuff into different folders.

Orderly - The folder structure in your primary archive should be a simple and orderly place. One of the easiest ways to structure this is to create a folder structure based on dates. Again, the Lightroom course shows how to create an archive in this manner.

Scalable - You expect to continue to make photographs, so you will want to create a primary storage structure that can grow with you. Once again, a date-based folder structure is a great way to make simple, orderly and scalable storage.

Use folders for storage, use metadata for organization

If you use a catalogue programme like Lightroom to manage your photo collection, you can take advantage of metadata to organize your photos in a lot of flexible and useful ways. It is easy to find all your photos from a certain date, or location, or all the photos for a particular client, even if they are contained in many different folders.

Once you start cataloging your photos and organizing them with metadata, it becomes a lot easier to work with a stable folder structure.

Figure 1 This movie discusses the difference between primary and backup storage.

Backup storage

In order to make sure you do not lose your photos, you need to make backups. Backups are simply an additional copy of the digital files that are stored on some other device, in case of a problem with the primary copy of the files. If you have created an orderly primary archive, then making backups can be a really easy process. There are plenty of software packages that can help you accomplish this, including free ones that are part of your operating system.

Rolling Backups vs. Disaster Recovery

There are 2 basic types of backups that should be part of your storage system. Ideally, you will have backups that provide protection against each of these.

Rolling Backups protect against the loss of current work, such as today's shoot, or the editing done in Photoshop today. Rolling backups should be created using software that can be set to run automatically. Ideally, this should happen at least once a day. This means that you need to have a backup drive that is easy to access, such as an external drive that lives on your desk, or in your laptop bag.

Disaster Recovery Backups protect against the loss of your entire collection, such as by fire, theft, lightning strike, etc. In order to do this, the backup copy should generally live off-site, in some other building. If possible, it is also good to make a write-once copy of the files for disaster-recovery backup. If you burn the image files to DVD, for instance, you can protect against a virus that might erase all hard drives on your system. The movie in Figure 2 describes the difference between these two types of backups.

Figure 2 This movie discusses the difference between rolling backups and disaster-recovery backups

3-2-1 Backups

A great rule for backing up your photo archive is to follow the 3-2-1 rule.

We recommend keeping 3 copies of any important file (a primary and two backups)
We recommend having the files on 2 different media types (such as hard drive and optical media), to protect against different types of hazards.
1 copy should be stored offsite (or at least offline).

While 3-2-1 storage is the ideal arrangement, it is not always possible, particularly for works-in-progress. In the next section, I outline how to think about backing up works-in-progress.

Figure 3 This movie discusses the concept of 3-2-1 backups.

Works-in-progress backup

The 3-2-1 backup outlined above is a great method to protect archive files. (Those are images that have been put into a permanent home on your Photo Archive drive). But what about works-in-progress? These includes images that may be temporarily stored on your hard drive if you are downloading in the field. Other works-in-progress might be your Lightroom catalogue, or multimedia projects that are not yet ready to archive.

For these files, a 3-2-1 backup is probably not practical. Burning new DVDs everyday as you work on a project is probably not possible, and getting an offsite backup of works in progress can be tough to keep current. The movie in Figure 4 outlines some ways to accomplish a works-in-progress backup.

FIGURE 4 This movie discusses how to properly backup works in progress

Storage hardware

Now that you have a better understanding of the entire storage picture, it is important to think about the hardware that we will be using to store the images. The best medium for primary storage is hard drive. And the best storage medium for backup is also hard drive. Optical discs, like CDs, DVDs or Blu-ray discs can be a great addition as disaster recovery backup.

In the next lesson, you will learn how to set up your storage hardware.

Online backup

In most cases, online storage will not be practical as a comprehensive backup for photographers. The cost for uploading, as well as the time it takes, makes it impractical for most. And while there are some services that offer unlimited uploads for very low prices, those are generally unsustainable business models. In January 2011 the service Mozy - one of the leading providers that offered unlimited storage - stopped offering that service because it was losing money.

Use online backup for your most important files

You can use one of these services for your most important images, however. And this can offer the added benefit of making your images more accessible to you, no matter where you are. For instance, a Flickr pro account offers unlimited storage of JPEG files, which can be 'published' right out of Lightroom. Currently, this service is only US$25 per year.

If you use ratings, as I suggest in the Lightroom workflow course, you will be able to select your best images and upload full size versions to Flickr automatically. And as you continue to refine your image selections, or as you make new versions of your favorite pictures, Lightroom can keep the Flickr account updated with all current versions.

There are other services as well, although they may not offer the same integration with Flickr that you can find with Lightroom.