I have a 56 TB local Unraid NAS that is parity protected against single drive failure, and while I think a single drive failing and being parity recovered covers data loss 95% of the time, I’m always concerned about two drives failing or a site-/system-wide disaster that takes out the whole NAS.
For other larger local hosters who are smarter and more prepared, what do you do? Do you sync it off site? How do you deal with cost and bandwidth needs if so? What other backup strategies do you use?
(Sorry if this standard scenario has been discussed - searching didn’t turn up anything.)
Tape or backblaze
I’m not sure if I qualify as a ‘larger local hoster’ but I would go through your 20 TB and decide what really is important enough to backup in case the wheels fall off. Linux ISOs, those can be re-downloaded, although it would take a bit of time. The things that can’t be readily downloaded such as my music collection that I have been accumulating for decades, converted to flac, and meticulously tagged, can’t be re-downloaded. So that is one of my priorities to back up. Pictures, business documents, personal documents, can’t be re-downloaded, so that goes on the ‘must back up’ list…and so on. Just cull out what is and isn’t replaceable. I would bet that once you do that, your 20 TB will be a bit more slim, and you’re not trying to push 20TB up the pipe to a cloud backup.
I use BackBlaze’s Personal, unlimited tier for $99 USD per year, which is a pretty sweet deal. One thing about Backblaze to remember is that the drives being backed up must be physically connected to the PC doing the backup/uploading. I get around that because I have a hot swap bay on my main PC, but there are other methods and software that will masquerade your NAS or other as a physically connected drive.
Backblaze personal doesn’t support Linux or BSD, so it would be useless for a NAS.
There are many ways to skin the cat. Here’s just one:
This Docker container runs the Backblaze personal backup client via WINE, so that you can back up your files with the separation and portability capabilities of Docker on Linux.
It runs the Backblaze client and starts a virtual X server and a VNC server with Web GUI, so that you can interact with it.
https://github.com/JonathanTreffler/backblaze-personal-wine-container
There are also other apps that will ‘fool’, for a lack of a better word, Backblaze to think a NAS drive is physically connected.
better would be something that can just eat a zfs send stream, but I guess for an emergency it’s fine. but I would still want to encrypt everything somehow.
I use backblaze too, started with the personal back up, but swapped to the B2 solution as it was supported by my NAS. The cost of the actual storage isn’t much, most of the cost is in access, so for data that doesn’t alter much it worked out just as cheap, and easier to do things that way.
and easier to do things that way.
I’m cheap and my labor is free. LOL But you do have a point.
The cost of B2 storage is very high, what are you talking about? USD$6 per terabyte per month would be like $4k a year for me.
Backup to 2nd nas.
Important stuff gets backed up to cloud storage. Whatever is cheapest.
In my case Synology c2 cloud was cheapest.
c2 seems expensive, I would go with hetzner storage box + restic
It offers some other features like hybrid access to data,If my nas isn’t available I can access it from their cloud. There’s also some identity services.
I’ve started using k8up to save my photos and config to an encrypted restic repo in an s3 bucket. having a lot of trouble backing up my SQL DB though, not as easy as they make it sound.
entire nas (~24TB used) is replicated to another nas in another building (2 actually). i like having 3 copies.
Similar to most responses, I backup whatever I created myself, not shared by someone or downloaded from somewhere. I care about pictures that I took, documents, financial records, etc, which don’t take up much space at all.
I’ve been following this post since the first comment.
And I have just put together my own RAID1 1TB NAS. And I did not think that 1TB will serve me forever, more like “a good start”.
But the numbers I’ve been seeing in here… you guys are nuts 😆
Definitely following this
What’s your recovery needs?
It’s ok to take 6 months to backup to a cloud provider, but do you need all your data to be recovered in a short period of time? If so, cloud isn’t the solution, you’d need a duplicate set of drives nearby (but not close enough for the same flood, fire, etc.
But, if you’re ok waiting for the data to download again (and check the storage provider costs for that specific scenario), then your main factor is how much data changes after that initial 1st upload.
I have 2 500GB SSDs in RAID1 for important data, truenas apps etc…, then 32TB total in RAIDZ1 for large Dataset that won’t need speed (movies, TV show, music, pictures, archives,…)
If I have a complete NAS failure, a remote backup (via rsync to a friend’s NAS Weekly) of the SSD and bootable drive can be used in a new system, and my torrent app has the list and magnet of all torrents stored on the SSD so it can re-download them.
So you have 56TB of total storage, but how much of that 56TB is actually used? Take the amount of storage used and add 10-12% to that figure. Now you create a new NAS (preferably off-site) with that amount of storage and that becomes your backup target. Take an initial backup (locally if possible to speed up the process) and then you can use something like rsync to create incremental backups going forward. This is the method I’ve used and so far it has worked out well. I target 10-12% more than the amount of used storage for my backup capacity because my storage use grows reasonably slowly. If your usage grows faster you might want to increase your “buffer” a little more so that you’re not having to constantly add drives to your backup target.
Yeah, this is certainly a viable “brute-force”-ish ooption. While I have 56, I’m only using 26 or so. But I’d actually be hesitant to do anything less than a full capacity mirror because I do expect to eventually use this (and more - adding drives to Unraid).
I’ve balked because of cost and upkeep (maintaining the same capacity, additional chances for drive failure, two separate sites I need physical access to with a high bandwidth connection), so I admit I was hoping I was missing an easier option.
I mean, if you want a full mirror, rolling your own backup target is going to be the cheapest option even with the current high price of hardware. Other options are cloud storage, or using another media like tape. Cloud storage is of course an on going cost which rules it out for me, not to mention privacy concerns. There are certain “cold storage” options from cloud storage hosts which are considerably cheaper but they have limitations on how the data can be accessed and how often. The tape route is possible but it’s not really viable for home use due to the high upfront cost of the drives. Outside of that, backing up a subset of your storage as others have suggested is the only other option. Creating viable backups without breaking the bank is a challenge as old as computers, unfortunately.
Backblaze offers unlimited data on a single computer, $99/year.
There might be some fine print that excludes your setup but might be worth investigating.
Oh shit.
only windows (maybe mac)
Yeah, people have done workarounds and stuff to get their entire NAS backed up but those seemed sketchy and bad when I looked into it.
if you break their TOS, you’ll likely lose your data. So… be careful. Mind you, I haven’t read their TOS, so i don’t know if those work arounds are breaking their TOS.
Wine or there is a Docker container that runs the Backblaze client.
Well, first while raid is great, it’s not a replacement for backups. Raid is mostly useful if uptime is imperative, but does not protect against user errors, software errors, fs corruption, ransomware or a power surge killing the entire array.
Since uptime isn’t an issue on my home nas, instead of parity I simply have cold backups which (supposedly) I plug in from time to time to scrub the filesystems.
If a online drive dies I can simply restore it from backup and accept the downtime. For my anime I have simply one single backup, but or my most important files I have 2 backups just in case one fails. (Unfornately both onsite)
On the other hand, for a client of mine’s server where uptime is imperative, in addiction to raid I have 2 automatic daily backups (which ideally one should be offsite but isn’t, at least they are in different floors of the same building).
Not all data is equal. I backup things i absolutely can not lose and yolo everything else. My love for this hobby does not extend to buying racks of hard drives.
True words of wisdom here from a self hosting perspective.
The stuff that I actually care about are automatically backed up twice, once to a simple external on site and once to a cloud. The cloud rotates between the most recent backups so it never takes up more than 1tb compressed, while the local external keeps backups for much longer (something like 6tb at a time).
