Data Storage

Move along, nothing to see here!
Seneca
Posts: 915
Joined: Sat Nov 24, 2012 4:58 pm

Re: Data Storage

Post by Seneca »

lilacorchid wrote:...Hard drives fail. Usually you get a lot of warning if you know what you are seeing, but not always.
Well, HDDs do tend to fail pretty gracefully and with warning. SSDs do not. Data recovery from a failed HDD is also more likely. Since so far SSD failure rates are not much better in the real world, and HDDs cheaper, HDDs still win provided it's not a rough use environment.

Losing your data from a hardware/software failure in the cloud is orders of magnitude less likely than if kept on a physical drive or two. Someone else looking at your data much more likely. There is also "black swan" systemic risk you are now open to. It is not likely a cloud provider bar access to your data, this is after-all the absolute core of their business, but they might want money.

Pick your poison...

slowth
Posts: 17
Joined: Mon Oct 21, 2013 3:25 pm

Re: Data Storage

Post by slowth »

henrik wrote:Ok, you're right about access. But like you said, that's why you have multiple backups:)
I did consider this, and I initially thought having a backup in the cloud was like having no backup at all since you don't have complete control. On second thought, having an encrypted backup (on your end, please my friends) in the cloud as an additional backup layer would not be such a terrible thing. I went off the deep end earlier saying it's idiotic, but I wouldn't dare use it as my only backup.

George the original one
Posts: 5406
Joined: Wed Jul 28, 2010 3:28 am
Location: Wettest corner of Orygun

Re: Data Storage

Post by George the original one »

And now we're on the same page :-)

dragoncar
Posts: 1316
Joined: Fri Oct 29, 2010 7:17 pm

Re: Data Storage

Post by dragoncar »

I run OpenIndiana via a virtual machine. I mount a "primary" USB drive on ZFS (which uses per-block data verification). I create a snapshot of the current volume and then synchronize data using rsync.

The USB drive is one of three. I keep two at home and one at work. Every month, I bring one of the "secondary" drives from home to work and synchronize to the "primary drive". I bring the secondary drive home.

This, way I always have at least one backup both at home and at work. This way, if either location burns down, I have the other location.

When ZFS reads or writes a block, it performs data verification (checks the block against a checksum) to ensure no data corruption has occured.

Yes, this requires a bit of extra effort to set up, but the backups themselves are relatively painless. Since I usually scan and then shred important documents, it's very important that my risk of loss is basically limited to a massive disaster (home and work both destroyed at the same time).

If you're more serious, use a third backup disk and send it far away.

KevinW
Posts: 959
Joined: Mon Aug 02, 2010 4:45 am

Re: Data Storage

Post by KevinW »

The paranoid solution is to protect against every imaginable threat by having backups that are diversified across many axes:

some local (easily accessible) vs some far away (won't get hit by same natural disaster)
some on network (easily accessible) vs air gap offline (immune to network attack, lightning)
diversify across multiple media (e.g. flash, spindle, tape)
diversify across operating systems (protects against OS-specific bugs or exploits)
diversify across countries (protects against country-specific war, terrorism, confiscation)

This line of thinking can go on endlessly and it's really up to you how far you want to take it.

Me, I rotate between 3 USB hard drives. One at home, one at work, and one unplugged at my parents' house thousands of miles away. That achieves decent coverage of the above factors without being a burden.

anomie
Posts: 442
Joined: Sun Apr 29, 2012 2:13 pm
Location: midwest, usa

Re: Data Storage

Post by anomie »

This seems like appropriate thread to mention to those hesitant over using Google Drive. Google at least lets you make a backup of and download ALL of your Google data now via Google Takeout

Google Takeout
https://www.google.com/settings/takeout
Your account, your data.
Download a copy.
Create an archive with your data from Google products.
This does not address concerns over what they do re: data mining, etc. with your data. But they do let you have a copy.

Google has always been more open than other big corp's re: options (I am thinking Gmail allowing you to POP to mail clients; my in-laws who use a gmail account as primary email still do not understand clearly that they can use web interface to Gmail to manage their email ( more effectively imo if they would learn it) instead of using Microsoft Outlook. They have been POP'ing their gmail to Outlook for 7 years..)

anyhow, hth someone.

arrrrgon
Posts: 154
Joined: Thu Aug 09, 2012 10:42 pm

Re: Data Storage

Post by arrrrgon »

For my personal stuff I really don't make backups.

At work I'm in charge of everything IT for four businesses. Three of those businesses are owned by my boss, so I control the systems. I use 3 methods for backup. We have a NAS that backs up everything once a week. I have a nightly backup to a hard drive dock, so I can grab the hard drive and toss it into a protective case to take home with me (all encrypted of course). The third backup is a bit of idiot proofing I use. Every now and then someone will delete something they need, but they won't realize it for a month or two. At that point the data is gone for most backups. I handle this situation by using a backup server that copies all files from our main servers to drives on the backup server. Rather than syncing with the original server, I just use xcopy. This means that files deleted months or even years ago are still on that server as long as we have hard drive space.

The servers use mirrored drives for redundancy, but I've had both drives fail at the same time before. The backups definitely come in handy then.

anomie
Posts: 442
Joined: Sun Apr 29, 2012 2:13 pm
Location: midwest, usa

Re: Data Storage

Post by anomie »

Look ma, no cloud!! ---

Last winter I became concerned about the fact that I was storing some work files on the cloud ; these files included passwords, other stuff. So I looked around for easy syncing options - methods to keep copies of data in synchronization between multiple computers. with no cloud middle man.

I found 2 great options:

Unison
http://www.cis.upenn.edu/~bcpierce/unison/

Open source project run by a UPenn comp sci professor. Has clients for windows and Linux; can set it up to run automatically via cron or scheduler.


and
Bittorrent Sync
http://www.getsync.com/

Created by the makers of Bittorrent, this software has nice clients for Linux, Windows, Mac, and mobile apps. You set it up, and forget about it.

Post Reply