Any suggestions for an efficient backup system?

Post Reply New Topic

So, in the past I only backed up to my external HDD whenever feeling like it, manually... now that the drive isn't very functional anymore I don't even do that, and whatever backup there may be is years old.

My problem is one with efficiency... copy/pasting everything again and again obviously isn't so, as it takes time, and I assume there is the possibility out there to automatically backup only files that have changed. I've read somewhere that rsynch could do this... but is it a complex procedure, or could it be fully automated fairly easily?

Anyone already has any such backup system in use by the way?

I don't really consider external HDD to be "backup" because in case the ceiling caves in or there's a fire, both your backup and original will be gone.

However for your specific issue, there's a Windows program called Synctoy v2.1 .

It is reliable, sturdy, and can run from commandline on a schedule. It can copy only new&changed files to your "backup".

The only fie safe backups are offsite backups. Perhaps something like Carbonite (tm).

ruveyn

Exactly. Not only that, the external drives seem to be less likely to last as long as an internal drive because they tend to take more abuse and because they generally don't have much in the way of cooling.

I know one woman with quite a following on the internet who was using an external hard drive for a backup. When it failed, she contacted the manufacturer and asked for help getting her data off the drive. Their response was to tell her she was stupid for using an external drive as a backup (not quite those words, but same thing).

I will only recommend on-line backups as a secondary backup and then only if the person/company has a good deal of bandwidth or if the amount to be backed up is fairly small.

For most individual users and small companies, there is a much better alternative.

Back up what you must back up to CDs or DVDs and store them in a safety deposit box well away from the computer. For people with PCs, there are some quite decent backup packages that are remarkably inexpensive. I haven't kept up with what is available these days, but I used to recommend Backup My PC for windows machines regularly.

With Backup My PC, you could put a CD or DVD in the machine and have it do a backup of what has changed during the off hours. Then once a week, do a complete backup of everything that you need backed up. Depending on how much data you have, you might just need to change disks once a week.

Of course, you don't want to waste time and space backing up the operating system. Just back up what needs to be backed up.

Although I greatly prefer Unix and Linux, I have yet to see a great hands-off backup for these.

... from 19th century.

Sorry, but Carbonite and Backblaze do a pretty good job with massive backups, and it's certainly a lot more convenient to set it and forget it than to back up 200gb onto what is at best 20 discs.

There's also the advantage of backup being much more up-to-date.

Both Carbonite and Backblaze use incremental backup systems, so only the first backup takes a long time. A "good deal of bandwidth" is not needed, only an ISP that doesn't cap your bandwidth.

Try using that anything like that much bandwidth around here and you will be at the very lowest priority. It would likely take you at least three months to back up 200 GB.

You would be "encouraged" to go elsewhere before that point, anyway.

Unfair? Not hardly. Around here, just the bandwidth cost to the ISP for you to transfer 200 GB would run about $800.

A 3 Mb (megabit) connection through the ISP where you can use all of the bandwidth it will provide would cost you in the neighborhood of $1,000 / month, payable three years in advance before the circuits are ordered with no possibility of a refund. Or you can deal directly with the telephone company.

In a big city where bandwidth is cheaper, it would certainly cost less, but you aren't going to make friends with the ISP using that much bandwidth. I've heard of network connections being canceled for using that much bandwidth.

Your problems are limited to your area. They do not apply indiscriminately to the rest of the world.

There are ISPs that actually advertise unlimited bandwidth and offer exactly that, without any tricks or backstabbing, and using stuff like Carbonite on them is absolutely a no-brainer. You cite pretty ridiculous figures. I pay $50 per month for such an ISP and have 4 computers in the house running Carbonite, 3 of them around 60gigs of backup each, and one around 120gigs.

You also have an incorrect understanding of bandwidth. Even if you're unfortunate enough to be saddled with an ISP that caps your bandwidth, you can still make sure that your backup program doesn't expend it all in one month. The initial backup will take longer, but inevitably it will catch up, and the incremental backups that follow don't take much bandwidth at all on an average office computer.

I understand bandwidth quite well. I did make a mistake, though. The bandwidth costs to the ISP for you to transfer 200 GB around here, it would be about $250 to $300 around here, not $800.

My advice will run contrary because I see things from a security and utility point of view.

Online backups are NEVER secure. If it's just photos and other nonsense that you don't worry being hacked or stolen, it's probably a good deal FOR YOU, but if your data needs to be "secure," none of them can assure they will never be hacked, the data never intercepted, or that nobody can access their data stores. In this era of NSA spying now being confirmed by Snowden, proceed accordingly.

Second, online backups are only optimal for backing up personal data. You would not want to image a whole HDD (or multiple HDDs) to the cloud. Massive bandwidth and storage requirements.

In my office, we have a server (single rack...size of a small coffee table). That server has multiple hard drives, and even then, we weekly image the main HDD to an external drive that is taken off site for storage. This server, if it suffered a failure, has multiple redundancies for restoring lost data. At no time are we using an online backup solution for our data.

I use external HDDs exclusively. I have NEVER suffered an HDD failure. Yes, it can happen, but for the cost of most online services, you can easily by two good-sized external HDD units.

With an external HDD, you can use a HDD imaging program to copy your entire HDD in compressed format. If you suffer a massive failure, you can get a new HDD and restore it in no time. A quality imaging program can do incremental backups (does a full backup every so often then routinely backs up CHANGES to that full backup).

When I set one up for a small office, the main PC had 3 HDDs. The main one (had the OS and programs on it), the "data" drive (all work files), and a third drive solely for backups. The data drive was imaged every weeks to the 3rd drive, and I tried to do a full system (OS and data drives) backup to the external HDD.

At home, I do something similar. I will run the image program of my HDDs and then copy them to an external HDD for safety.

The best way to avoid external HDD failure is to not buy a crappy HDD. I shop places like Amazon and NewEgg and read the product reviews by customers. Not a perfect system, but if you see a drive gets negative reviews, it's a good clue to not buy it.

I forgot to mention that I used to write backup software.

One other thing regarding on-line backups -- it is crazy to use on-line backups if you don't fully understand the backup policies of your particular backup provider and the limitations.

For example, many people think that they can delete files off of their drive and retrieve them when necessary from the backup provider. In fact, every backup provider that I know of will delete those files from the backup after a certain period of time, generally two to four weeks. The logic is that since it is gone from your computer, it must be for a reason if you haven't restored it within that time.

Another problem with online backups is that if you lose your drive and have to restore all the files to a new drive, you're looking at a rather extended period of time if your bandwidth isn't great. If you lose your hard drive and need to restore hundreds of gigabytes of data, do you mind it taking several months?

And one often overlooked problem is about what to do if the backup company shuts down the service. A couple of years ago, one company that offered online backups as an additional service did shut down. They put a notice on their web page that they were doing so and that the data would no longer be available after a certain date. Most customers never saw the notice until it was too late. Those who needed to restore from the backup after that date (or who were restoring when the data was taken off line) lost everything they had not restored by that time.

Some people have used other services such as Megaupload to store data. Guess what happened to their ability to access that data when the government shut them down? That's right. Complete loss of data.

Good points.



We've lost several.

I lost one drive that was used to make a copy of data that wasn't particularly important.

One local photographer had a computer with a number of external drives attached for storage of his pictures. He lost at least three of the drives within about a three month period. And they were good drives.

The reviews on places like Newegg might provide some good advice, but they don't really help to evaluate the time the drive will last.

For years, the gold standard for backups has been tape drives. Unfortunately, they get to be rather expensive. I understand that some places such as museums that need archived data to be available practically forever are moving toward DVD storage with the DVDs stored in a very carefully climate controlled room. They are expecting to get at least 100 years out of DVDs with that method.

No, sorry. I have no idea of your level of computer proficiency, but one if you're reduced to copy+paste, you can speed the experience up by using keyboard shortcuts.

CTRL+a :selects all
CTRL+c: copies
CTRL+x: cuts
CTRL+v: pastes

If you already know this, sorry for wasting your time. If you don't, it will help you speed things up a little bit. FWIW, this is the way that I've ben backing my files up for the past several years.

The up side is that you will still have all of your files on hand if you screw up your system, up until the last time you saved your files.

The down sides to this of course is that you will only have the files on hand from the last time you restored, which could be problematic if you havn't backed up in a while, and any software that you have as well as software/OS configurations that you are running will be lost if you have to format and reinstall the OS, which means that you will have to reinstall/reconfigure software and OS afterwards. --Again, sorry if this isn't a help to you, or if you already know this.

If you make a duplicate then it's a backup regardless if the duplicate is stored in the same location as the original. I too use a Seagate USB hard drive to back up my user profile data however I created a startup script that searches for the hard drive id and name then starts to automatically move data over. Using the hard drive id and hard drive name that helps protect against automatically backing up to just any USB device plugged in. If you need it I can provide you with my script and you can modify it as you see fit. You may need a little understanding in vbscripting though.

Not sure why you agree? Data centers have backup tapes in the same rooms as the servers that they've backed up. Yes worse case scenario is the room caves in and both the original and backup are gone but REALLY how many roofs have caved in on you? Backups are for the event that a hardware fails beyond the point of recovery.

Online backup is not practical if the amount of data that needs backing up is large relative to the available bandwidth - and that's even before you get to storage costs.

Local backup, where the backup media is physically located in the same place as the computer, is much more practical. Get some sort of RAID controller - at worst you can use low-level RAID which automatically just mirrors one drive to the other - more elaborate RAID schemes exist where you have 4 or five hard drives in a RAID array and the data is automatically backed up across the drives such that if any one drive fails, all of your data remains intact - of course, with these schemes, the total disk space that you purchase is greater than the amount of disk space you can actually use (if you've got one drive mirroring the other, for instance, your usuable space is only half of what you paid for). Of course, the disadvantage to local backup is that while your data may survive the failure of any one disk, it won't survive some sort of disaster that destroys an entire location - so at the end of the day its a risk/reward/cost calculation and compromise.