Your Hard Drive and its Data
Avoiding Hard Drive Crashes and their
A marvel of modern
engineering, yes. But an ultra-reliable device you can
bet your (data) life on? No.
Hard drives remain prone to
problems and you never know when they mightn't suddenly fail
and destroy your data.
Hard drives have greater and
greater capacity, and all for less and less cost, every day.
But their underlying reliability
- the MTBF (Mean Time Between Failures) seems little unchanged
from a decade or more ago. The new high capacity hard drives
equate to a greater headache in terms of data backup and
subsequent data recovery after the inevitable hard drive crash
Fortunately, there are now some
excellent solutions that - at long last - makes backup easy and
foolproof, and some prudent precautions you can take to anticipate
a hard drive crash and solve the problem at a time of your
choosing rather than when the hard drive decides to suddenly fail.
Protecting Your Data is Both Easier - and More
Our data storage 'needs' have grown,
probably in line with Moore's Law (ie doubling every two years or
so). Most people now have a 100+ GB hard drive in their
computer, whereas a decade or so ago it would have been a 100 MB hard
drive, and a decade or so before that, hard drives were still an
optional upgrade rather than an essential part of a computer.
Although the cost of drives are now
unbelievably low (in terms of cents per Gigabyte) and the size
of the drives unbelievably small, the reliability issue remains a
weak point - and indeed this is probably because of the other two
Even good hard drives
'fail' thousands of times a minute. They misread data all
the time, but most of the time, they are able to correct these
errors without pausing in their stride. It is only when
their performance degrades beyond the built-in levels of fault
tolerance that the problems become noticeable and threatening.
For a fascinating 'look under the hood' the
video on this site
is interesting, albeit technical.
If manufacturers thought they could justify a
higher price in terms of more reliability/longevity, they might be
motivated to do so, and if they thought that people would accept a
more bulky drive that was more reliable, again they might do that,
but their feeling at present is that hard drives are 'throwaway'
items that are best produced as inexpensively as possible.
Sure, hard drives may be throwaway in terms of the
cost of buying a new one, but in terms of the inconvenience of
losing the data on a drive (and the hassle factor of reinstalling
all your software on a new drive), they are enormously expensive and very undervalued items.
Of course, the more data we store, the more
data we must backup and protect. Fortunately, the
old-fashioned ways of backing up data - ways which were
cumbersome, time consuming, expensive, and relied on us to
voluntarily participate in the process - have been largely
replaced by convenient automatic and inexpensive solutions that
now finally and truly leave us with no excuse not to have our data
all thoroughly backed up, all the time.
Read on for how to manage and control your
own data storage.
RAID - Safety in Numbers
If you have a laptop, you probably only
have space for one hard drive in your computer. But if you
have a regular desktop style computer, you probably are vaguely
aware that inside the computer's case there is room for two,
three, four - maybe even more hard drives. However, the
chances are you only have one hard drive installed - with the
drives being so huge in capacity these days, who needs two, right?
It is true that in the past, there were
capacity and performance reasons for having more than one hard
drive in one's computer, and it is also true that these days, both
issues are of less importance. However, there is now a
reason for having two or more drives in your computer that takes
advantage of the ever-lower costs of hard drives and related
This reason is to have an automatic backup
system whereby all your data is written onto two drives
simultaneously. If either of the two drives fail, your
computer simply swaps over to the other drive and asks you,
politely and non-urgently, to please replace the failed drive one
of these days.
The replacement is also an angst-free
operation. Pull the old drive out. Plug the new drive
in. Nothing to reload or reinstall or recover.
For you as the computer user, all the magic
of the process is completely hidden. Everything is the same
to you in terms of file management, programs, controls, and so on.
Apart from probably seeing one more message flash over the screen
when the computer is booting up, everything else will be the same.
This amazing ability has been around for a
long time, and in its most sophisticated form is referred to as
using a RAID device. A RAID (which stands for Redundant
Array of Inexpensive (less frequently and less correctly the word
'Independent' is sometimes used) Disk (drives) ) is simply two or more hard
drives connected to a controller card (which is built in to the
mother board of many computers these days, otherwise which can be
purchased and added to the computer subsequently).
controller does all the behind-the-scenes housework of checking
the drives and keeping duplicate copies of everything on both
drives, and of taking a failed drive offline with hopefully no
interruption of service or data loss.
There are different types of RAID array.
Some offer data safety, others offer faster disk performance, some
offer a mix of both. The one of most relevance for backing up data is also the
most inexpensive and simple form of RAID array, and is referred
to as a RAID 1 array - a pair of mirrored drives.
Your action item is simple : Next
time you buy a desktop computer, consider asking for it to contain
a RAID 1 controller and two drives rather than one.
This might add $200 to the cost of the
computer, but it will almost completely eliminate your fear
of future data loss from a corrupted hard drive. There is nothing
new for you to learn, no new complexities or software programs or
anything. It just happens, automatically in the background,
the same as the rest of the computer's hardware operations.
Automatic Off-Computer Backup
If you don't have a RAID 1 computer, you of
course need some sort of backup procedure. Even if you do
now have a computer with a RAID 1 capability, you still need
off-computer backup too.
Why is that? Isn't the RAID 1
supposed to solve all problems, make the coffee for you in the
mornings, and take the cat out at night? Alas, no. You
remain exposed to two potential problems - the first is a failure
in the RAID controller card itself that could, as part of its
failing, corrupt data on both drives, and the second is an
external type of problem (more about this in a minute).
Clearly, if you're not mirroring your data
on two drives in your computer, you have no backup for your
computer's data and must have some way of having an up-to-date
copy if everything, just in case something goes wrong. And
the list of things that could go wrong is broader than you might
Reasons Why You Need Off-Computer Backup
You need a copy of all your information off
your computer for several reasons. What happens if your
computer is lost or stolen? If your data is backed up on
your computer, then the loss of the computer means both your main
and your spare set of data have both gone too.
The same is true if instead of losing your
computer, it is destroyed in a fire or other awful event like
So your backup copy of your data needs to
be somewhere unrelated to the computer itself. At the very
least, it needs to be where a thief wouldn't find it and take it
as well as your computer.
Ideally if it can be
completely away from the location of the computer, then you can
have fire, floods, famine, or just about anything else where the
computer is (was) but still have a copy of your data somewhere
else, unharmed by whatever misfortune befell your main computer.
Remember the bad old days of setting a
schedule to back your computer up, and feeding it a collection of
floppy disks, one after the other after the other? Or
running a tape drive? Or burning CD copies?
All those processes are now obsolete and
have been replaced by much more convenient programs that will be
all the time monitoring your hard drive, and any time they see a
change to any file, they will then instantly take a copy of it and
shuffle that copy off to your backup device/location.
If you are not yet using an automatic
backup program, you should - no, not should. You must get
one and start using it as soon as possible. It is so easy
and simple, there is no reason or excuse not to do this.
The easiest type of automatic backup system
is to go to Costco or Amazon and buy a ridiculously huge external
hard disk that comes complete with the automatic backup software.
These drives can be either a local drive - ie, connected to the
computer by a short length of probably USB cable - or a network
drive (often called a NAS - Network Attached Storage - device), which can be located somewhere else in your house/office,
but connected to your computer via the network (either wireless or
Clearly, it is better to have the drive
somewhere away from the computer, in a non-obvious place so that
thieves would be less likely to find it and take it as well as
your computer at the same time.
A network attached drive has another
advantage too. You can have multiple computers all sharing
it for backing up purposes. So instead of needing to buy a
backup drive for each computer, you just need one and you can use
it for two or three different computers (different drives and
software have different limitations on how many computers they
will allow to share one networked drive).
If you are choosing the networked drive
option you need to
understand how many users the drive will allow to be connected to
it, and if the drive can be connected by Wi-fi or only wired
ethernet cable. And get a really big one - there's little
difference in price, and it gives you more ability to grow into
page on Amazon listing most of their NAS drives. I have
a 3TB Western Digital device myself.
The ultimate in backing up away from the
physical location of your computer is to back your data up into
the 'internet cloud'.
These days there are a lot of companies
that offer automatic backup services, copying your data off your
computer and onto one of their servers, somewhere else in the
country or world. If you have a fast internet upload speed,
this can be a great way of getting your data safely stored
somewhere else. (Note that stated internet speeds are usually based
on the download speed - your upload speed might be very much
slower, and can be
measured by a service such as this.)
Two established examples of cloud backup
services that operate automatically on your computer are
Mozy (nothing for free, their least
expensive package a 50GB allowance for $6/month) and
Sugar Sync (5GB for free,
30GB for $5/month).
Google, just a couple of weeks ago,
announced a product - Google
Drive - that gives 5GB for free and 25 GB for $2.50 a month.
So there we were, discussing a local backup
drive with 3TB (ie 3,000 GB) of capacity - that makes 5GB seem
awfully puny, doesn't it! Indeed, your computer probably has
a 100+ GB hard drive on it, and for sure, if you check, you'll see
it has way way more than 5GB of data currently on it. But
maybe you don't need to back up all that data.
How Much Cloud Storage Do You Need?
It is one thing to be choosing between your
own local storage device that has a 1TB or 2TB or even 4TB or 8TB
capacity, and with only a few tens of dollars separating the cost
of the smallest and the cost of the largest of these devices.
In such cases, obviously you are well advised to get more than you
need right now, because - as we all know - our data storage needs
have a habit of growing and growing.
But when you're having to pay some dollars
every month for every GB of storage online, you need to take a
more careful look at what you need to store. The good news
is you actually need to store much less than you thought.
First of all, there is no point in storing
anything to do with your operating system, because you can't just
copy all that stuff back to your computer if you need to.
You have to re-install it from your original disks rather than
copy the installed files from backup.
The same is true of most other programs.
Don't copy any program and configuration files for any of the
programs you have, because they all have to be re-installed too.
If you downloaded programs in installation
packages to then be installed, you would only need to keep copies
of these if they were protected downloads that you can't
re-download in the future for no extra payment.
Now for your actual data files. Most
people will find that the largest part of their data files are
videos, music tracks, and pictures. A single two hour HD
movie could use up your entire 5 GB of free storage somewhere -
or, to look at it another way, it would be cheaper to buy DVDs and
Blurays than to pay for the cost of remotely storing such movies.
We suggest that the large sized types of
files such as mentioned above - files which are probably seldom
accessed and never changed - be backed up onto some sort of local
device. Your cloud backup should only be for data files -
spreadsheets, documents, and those sorts of things, and your local
email files if you use a local email client rather than a cloud
You'll probably find that 5GB is more than
sufficient for these types of files (although if you're like me,
you might find that your old emails will spill over 5GB by
themselves - what I do there is I have old email archives of old
emails that I store locally, and only keep the ever-changing
current email file stored remotely).
The Downside to Remote Cloud Storage
There is a downside to using a cloud based
service as well. You are reliant on being able to access the
cloud storage service any time you need to - both for
uninterrupted convenient backing up, and of course, most of all
for being able to retrieve your backed up files in the future.
This makes you reliant on both convenient fast internet access and
the continued uninterrupted operation of the remote service.
There are some surprising risks and
downsides to this, well covered in this article about
the dangers of cloud based computing.
So, by all means use cloud based backup
services, but use them as well as, rather than instead of, your
own backing up procedures. Remember, a
bird backup in the hand is worth
two in the bush cloud.
Some people are also uncomfortable with the
privacy of their data. All the leading remote backup
services use encrypted communications for your data to be remotely
backed up, but the unavoidable fact is we regularly hear of credit
card companies having their 'secure' computers hacked and our
personal information compromised, so there is some element of risk
whenever we store information outside of our control.
Disk Health Checkups
Do you have annual checkups at your doctor?
Do you go to your dentist from time to time?
What about your car? Is that
regularly given lube jobs?
And what about your computer? When
was that last given some preventive maintenance? These days
many of us rely on our computer more than our car, and whereas it
is easy to rent a replacement car, there's no way we can rent a
replacement set of data if we lose it.
Fortunately, in most cases, there is
software you can use to test your hard drives and in many cases
these programs will warn of incipient failure. Most modern hard drives have some
built in error recovery capabilities, and by seeing how much error
recovery the hard drive is having to do, it is possible to measure
how well the drive is performing.
Hard drives also have
'spare' space on them that can be pressed into service when some
parts become 'bad' and being able to see the amount of spare space
that has already been used up gives another measure of the
condition of the drive.
There are three types of hard drive testing
The first is a program provided by
your hard disk manufacturer. Most hard drive
manufacturers will provide software to test their drives.
Look on their website, or if you can't find it, ask their support
has a list of major drive manufacturers and the disk utility
software they provide.
The second type of hard drive testing might
be provided by the manufacturer of your computer.
Usually the most rigorous tests are pre-boot tests, but maybe
there is a utility program that can be run after your computer is
up and running, too.
Again, if you can't find this software, ask
your computer's support people about it.
A third type of hard drive testing is an
independent third party program. There are
many of these, and some are free while others are not.
One of these programs stands head and
shoulders above all others. It is the
sold by Gibson Research. It does a better job of diagnosing
the health of your drive, of recovering bad data, and sometimes of
restoring your disk back to health again too. Very highly
recommended (and Steve Gibson, the developer of Spinrite, is one
of the all-time 'good guys' of the internet, having done
pioneering research on issues like spy way and security weaknesses
which he has generously released into the public domain).
One type of software which won't really
help you is that within your operating system (presumably
Windows). The Checkdisk utility won't really tell you if
your disk is alive and well, or on its last legs. Don't rely
on this to help you anticipate disk crashes.
Not to sound obsessive, but if you don't
choose Spinrite, you might want to use both your hardware
manufacturer's utilities and also your hard drive manufacturer's
utilities too. Sometimes one might detect something where
the other does not.
Note that you should always do the complete
or extended or advanced testing. The quick testing might
pick up serious errors, but it will not necessarily identify more
subtle errors that may be evolving.
How Regularly Should You Test Your
How often should you test your harddrive?
If Spinrite gives it a two thumbs up type
clean bill of health, we'd suggest maybe once every half year or
so (add it to your list of things to do with each Daylight Saving
As the drive ages, you'll want to keep a
more careful eye on it.
Any time your computer starts misbehaving,
you'd be well advised to include a detailed drive test as part of
your troubleshooting. Sometimes hard drive problems can
manifest themselves in strange ways, due to the complex
interaction as between a computer's hardware and the software
which controls it and the additional software which interfaces
between the computer and you.
You can protect your data,
within your computer, by adding a RAID 1 disk controller and extra
disk to your desktop but not laptop computer.
You can protect your data,
outside your computer, with a cable connected remote hard drive
nearby, or a LAN connected NAS drive somewhere else in your
You can also protect your data
completely off-site via a cloud backup service.
Automatic backup software
ensures your backups are always up to date.
These various different
approaches are all simple and easy and inexpensive. You no
longer have any excuse not to have 100% backup and protection for
your computer data.
If so, please donate to keep the website free and fund the addition of more articles like this. Any help is most appreciated - simply click below to securely send a contribution through a credit card and Paypal.
25 May 2012, last update
15 Oct 2013
You may freely reproduce or distribute this article for noncommercial purposes as long as you give credit to me as original writer.