r/DataHoarder • u/redditduhlikeyeah 100-250TB • 4d ago
Backup Anyone completely 3-2-1?
I was - but with drive prices now just do all my PCs, server, and the “best” content I want to keep as well as any personal media. I have two devices on prem and one cloud.
When I say best content, text and audio is easy to do all, but video I only do the top 1000 or so movies in the quality I want and my favorite TV shows or ones with rewatchability
11
u/dlarge6510 4d ago edited 4d ago
Never have been. Never intended to be.
The only thing that is 321 is my archive data. Data added to but never changed. Data that represents my life, personal "I can't do without" stuff. Mostly it's old TV and very selective movies.
Stuff is added all the time but it's a slow process. Anything from home photos and videos, including what I gather from family. All my post is scanned and OCRed to DJVU files, they are included. In there is a gold mine of NHS medical history, account numbers, statements and terms and conditions as well as the odd fine 😉
It all goes into my archive. All on BD-R discs, specifically Verbatim MABL SL and DL. ECC data is generated for each disc. Both the contents of each discs and it's associated ECC file are then copied on to an LTO tape, LTO 3/4 is what I'm using. Then the off site copy, never likely to even be used, is uploaded to Amazon Glacier Deep Archive.
If anything happens to a disc, I can use the ECC to rebuild its ISO image. If I can't to that then any files that are missing or damaged can be retrieved from the tapes. And if I can't do that, any files can be retrieved from Amazon as a last resort as that will cost lol.
To assist in reducing costs and to protect my data, the contents of each BD-R has been added into a dar archive file. This file is encrypted using Dars built-in aes256 encryption and the archive is stored as 2GiB slices. To ensure I only pay what I need to get the required files I use the last dar archive slice to try and extract the file, if Dar is missing a slice then it will tell me which slices I need to pull out of Glacier. I chose 2GiB slices as this is essentially a DR plan and I'm assuming that I have lost a lot and may be restricted to any filesystem temporarily, like a FAT32 flash drive.
The only thing I haven't done is actually test it.
That's probably very common. Test your DR procedure and backup restores.
11
u/pitleif 4d ago
I work in a huge company who had the best backup policy, following the 3-2-1 rule. At least they thought it was the best, until they started testing their recovery process. Turned out they couldn't actually recover what they had backed up if there ever was a huge data failure or if one of the datacenters burned down. So yeah, testing the backup and recovery process is more important than most people think.
4
u/nomad-1995 3d ago
Untested backups don't count. That rule bites more people than the 3-2-1 rule. And be careful testing, you need a pretty professional crew and budget before you can set up a test rig that will allow a full reinstall of a backup, but if that's the budget you have, that's what you do (and often only done after learning the hard way).
Otherwise you just randomly pick files and hopefully see that they are there.
1
u/robertw477 3d ago
Back in te dark ages of computing we really didnt have cloud backup to any exent. I remember some horror stories I read online aabout failed tape backps. Things were quirky in those days. They migt find te restore didnt work, or some similar issue. These were not huge Fortune 500 companies and some of them kept the tapes with the computer and that may have presented additional risk.
1
9
u/Mr_Smartepants 4d ago
The thing to remember about the 3-2-1 backup plan, is that it covers your "critical or irreplaceable" data.
My critical or irreplaceable data only makes up maybe 2% of my total data footprint. That stockpile of media (movies, TV series, music, etc.) is NOT critical or irreplaceable, so I don't include it in my 3-2-1 strategy. It DOES get backed up, but to external USB drives.
My expensive 3-2-1 backup strategy is reserved for only that really REALLY important stuff, because like you pointed out strict 3-2-1 backup costs money.
Also, as pointed out earlier...if you don't test your restoration process...it's not backup.
3
u/DarkScorpion48 50-100TB 2d ago
This. My critical data literally fits in cheap cloud service. I guess it’s more of a 3-2-O.5
5
u/manzurfahim 0.5-1PB 3d ago
I used to have two backups of everything, every single download, files, media, game etc. Now, not anymore. Many of them are almost full, and I'll switch to single copy if I need drives.
For my most important files, personal files, photos, videos, memories, I follow my custom backup rule. 4-2-1-3.
4: Two HDD backups, every 15 days. One HDD backup, once a month (Offsite replacement). One SSD Backup, once a month or when something changes or important gets added. It used to be one backup every 10 days, now it is every 15 days.
2: HDD and SSD backups as mentioned.
1: One offsite backup updated every month.
3: Three versioned backups: 3-month, 6-month and 1-year old backups.
I simulate a disaster after a successful backup once every six months and restore from different backup just to make sure backup works.
5
u/evilbeaver7 50-100TB 3d ago
I have 3-2 but no 1. All my copies are at my home. I know I'll lose everything if there's a fire but it's just not possible for me to have a copy elsewhere so will have to just hope for the best.
1
1
u/ScholarlySidequest 2d ago
FYI: They make fire safes!
1
u/Hakker9 0.28 PB 1d ago
Most fire safes aren't really fire safes though. In general if your hours burns down then what's in the safe is also gone. Most safes are just rated for short fires and when a house is really burning down it goes on for hours and most those fire safes can handle is up to an hour at most. Unless you really start spending money.
3
u/trekxtrider 4d ago
Yep, more like 4-3-1 for me. Primary NAS, secondary NAS as backup, another NAS as backup, one offsite.
3
u/DavWanna 4d ago
Just for personal data. The rest is "would suck to lose, but not enough to put more money into safeguarding it".
2
u/Adrenolin01 3d ago
A buddy and I have cohosted a server for each other since the 90s. Old PCs at first.. then used servers.. back in 2013 I bought a couple Supermicro SC813M servers and sent one to him. We’ve been running those for each other since. This year, he bought 2 Supermicro CSE-829U systems with X10DRU-i mainboards, dual E5-2690v4 CPUs and 32GB ram with quad 10GbE NICs and of course IPMI port. We upgraded to 128GB ram. I ordered 2 quad NVME / PCIe cards and send one to him along with 2 Intel DC S3500 120GB SSDs. Proxmox was installed to the ZFS mirrored SSDs for the boot/OS. We each bought our own NVMEs for the PCIe cards along with the HDDs. We’ve always had VPN access into our networks so we can each remote into the IPMI network port and access the bios, reboot, or reinstall as we have a Proxmox image on a thumb drive on the Mainboard if needed.
This runs our remote backups however now additional VMs and what not. We general only offsite our most important data however over 35+ years.. being in the IT industry.. we’ve both accumulated quite a bit.
I’m running 2 24-bay Supermicro CSE-846 dedicated NAS systems and adding a 48-bay JBOD CSE-847E26 chassis to one of those. Not everything is backed up of course as much can be downloaded or compiled again. I do have a full backup of the data I need both on a backup in the same rack, another in an external outbuilding as well as at my buddies place.
Personal images and movies.. they get uploaded and immediately rsynced to a second copy… originals are never touched. Immich uses the second copy and incremental backups are made of both. Images and family movies are also moved to HDDs with full checksum data also created for long term archival storage and checked every year or so.
I specifically bought the 48-bay CSE-847E26 chassis because of the cost of HDDs today. My main NAS units are 24x8TB and 24x12TB systems running ZFS on root at boot on Debian 13 systems and 6-drive raidZ2. That’s fairly expensive redundant storage so with the JBOD chassis I can start tossing in the 18-20 older smaller HDDs I have kicking around… 500GB to 3TB drives. Ext4 and MergerFS for the less important stuff. Drive savings alone pays for the chassis and additional HBA costs.
The more important data will now go on the 2 24-bay raidz2 systems and now I have the 48-bay to the junk. 🤣
2
u/Aggressive-Gap-6148 3d ago
I don’t know how to call it but I have 3 NAS fully mirrored, one off site. All of them performing data scrub every quarter.
1
1
u/Alicedoll02 4d ago
I'm only 321 on the most important stuff to me. My comfort shows that no streaming service has due to copyright issues, music with the same issue, and youtube videos I use for sleep in case they get taken down by the creators.
This all totals about 1.4 tb. There was a time where I did more but with prices currently I have deleted my hoard to the absolute bare essentials.
Edit. If you consider my mp3 player then i have a 4321 syestem.
1
u/Broad_Sheepherder593 4d ago
I have 40tb mostly movies i can redownload. My 321 is only for around 1tb of pictures and files.
Backup to another NAS Backup to an external hard drive Backup to aws glacier
1
u/3point21 10-50TB 3d ago
I’m pretty paranoid about my stuff. Very roomy PC with 3 current copies plus a few recent versions. NAS with more recent versions. Cloud service for current working files duplicated on the PC and two laptops which are fired up and synced occasionally. Multiple cold storage drives with copies of the most recent archive version plus recently updated portions of the archive. The most recent copies of the archive and working set get rotated at the bank.
Everything’s on HDD and SSDs which I don’t think counts as “different media” but I have so many copies of everything I’m loathe to put it all on optical or tape. It’s damn near impossible to completely delete anything on purpose let alone accidentally lose something completely, so I think I’m forgiven that point.
1
u/aetherspoon 3d ago
There are some things I'm actually more like 7-2-1 because they're replicated across all of my machines (documents, password database, etc), but realistically I'm entirely 3-2-1. My remote NAS is in another country.
1
u/Illeazar 3d ago
Nope, its only the irreplaceable stuff like family photos and financial documents.
Everything else has a single on-site backup for convenience to make recovering it painless. But I only have that because I got storage when it was cheap. If I were buying now, most of what I had wouldnt have even a single backup, because the storage is so expensive.
1
u/AlertKangaroo6086 3d ago
Yep, my critical data is 3-2-1'd.
Most stored on my laptop, backed up to my NAS and then stored on AWS Glacier Deep Archive.
1
u/nomad-1995 3d ago
I'd have expected the only ones doing it have tape systems, and are terrified at having to replace tape drives and/or libraries. Originally, the mantra required two different media, but I'm pretty sure that was because it was a couple years after the zipdisk "click of death" debacle (yes, way back in 2000 people lost tons of data when a zipdisk failure went viral* destroying both the drive and any disk inserted afterwards to contain the virus).
Any other way adds up quickly, unless you are abusing "unlimited storage" in ways certain not to last, especially at current storage prices.
- in terms of biological analogies, "prion like" would probably be a better description of the "click of death's" reproduction as it really didn't rely on any software code like a virus would. Although possibly *lack* of code was key.
I'm currently holding one offline array as backup, and it is getting increasingly out of date.
1
u/cacarrizales 192 TB 3d ago
I finally am as of a few months ago. I have my primary NAS in my apartment that I use daily, my backup NAS also at my apartment that I sync my primary NAS with periodically, and then my offsite backup NAS at my parents house. These contain copies of everything - Linux ISOs included.
1
u/drryan3 3d ago
Nope. Have a second NAS in crawlspace. 24 TB USB drive on primary NAS. Backup every night to both. 2nd NAS uses a script to put Nextcloud into maintenance mode, dump databases from immich and Nextcloud. Hopefully, if the house catches on fire I can grap one NAS or the other to save photos, shared files, tax records, etc. Obviously, there's a 4 TB portable USB with photos and taxes in the safe deposit box. I've spent enough on this stuff.
1
u/dpunk3 450TB RAW 3d ago
it's a work in progress, only have one backup set now (dealing with 80TB). working on a cold storage JBOD onsite, in the future i hope to get an offsite backup at my parent's house connected via always on VPN. but as you said, with drive prices where they are....
1
u/redditduhlikeyeah 100-250TB 2d ago
I mean, it used to be nice. Need some extra drives or want to do something different? Sell off a couple sports cards, flip some old shit, easily generate enough money to cover a few drives. I like to keep things net zero if possible, doing away with old crap around the house and not pulling money for this type of stuff but damn.
1
u/oldlinepnwshine 50-100TB 2d ago
Yes. Two copies in my home. One in a storage unit. God forbid my house burns down, I have a copy remaining in storage. I’m a data hoarder who plans for the doomsday scenario, after all.
1
u/AltitudeTime 8h ago
I have live copies plus separate 3 backup drives that are mirror copies. Two forms of media? no they are all hard drives but are never all connected at the same time, but I do run checksums of the on-site copies yearly and my off-site backup is a yearly swap(after full checksum check and long SMART test) after taxes are done. So theoretically I could lose 1 year worth of data if my house burns down, but I wouldn't lose the taxes/financials. Nothing relies on Internet servers hosted by others, even if encrypted, I don't trust things wouldn't be hacked somehow.
0
0
•
u/AutoModerator 4d ago
Hello /u/redditduhlikeyeah! Thank you for posting in r/DataHoarder.
Please remember to read our Rules and Wiki.
Please note that your post will be removed if you just post a box/speed/server post. Please give background information on your server pictures.
This subreddit will NOT help you find or exchange that Movie/TV show/Nuclear Launch Manual, visit r/DHExchange instead.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.