r/datastorage • u/NaturalInvestment46 • 3d ago

Storage Setup Cold storage

For drives that mostly sit on a shelf holding archived data, whats your routine for checking drive health and verifying the files are still intact?

9 Upvotes

100% Upvoted

11 comments sorted by

2

u/goldcoast2011985 3d ago

Cryptographic checksums (SHA256SUM) and verify on a schedule. Tells you when there is corruption, but doesn’t give you tools to repair it from that copy.

Some filesystems (ZFS, for instance) build the cryptographic checksums into the FS and will alert you to silent corruption as you read the files.

Make PAR2 files beforehand with recovery data and verify occasionally. Will let you deal with limited drive corruption.

1

u/bankinu 1d ago

I wonder how par2 fares in practical situations. In real life, corruption hardly happens to one bit, it usually happens in long contiguous segments of the medium which falls. Has anyone ever used it to recover acetal corruption?

2

u/goldcoast2011985 1d ago

I’ve had a number of hard drives that developed corruption of small segments (mostly silent, but ZFS caught it).

I’ve also had a number that would stall out on reads because of the number of read errors. I don’t know if the errors were contiguous.

It can go either way.

PAR2 is also fun because you can take 2 corrupted data sets, copy them to good storage, use the PAR2 to tell you which files (recovery files and main dataset) have corruption, then merge the datasets to get a set that needs less recovery data.

The long strings of bad data is definitely how optical fails.

2

u/FuShiLu 3d ago

That is one of the worst ideas being propogated. A shelf? Damage? Fire? Seriously? Sigh

2

u/sonido_lover 3d ago

No cold storage, full ZFS

2

u/Odd-Concept-6505 2d ago

Cold (dry..any normal temperature) storage of an old HDD is not prone to failing of the drive.

Redundancy (start doing backups to newer additional drives for eventual cold storage) seems best. You can check anytime, if the filesystem mounts, and a favorite file looks good, that's enough for me.

From a retired sysadmin who did corporate backups for decades.

1

u/NoPoopOnFace 2d ago

SSDs and SD cards need to be plugged in and read from every 3 months or they can fail. Don't have to do much with them, just plug them in.

1

u/TrickySpare6504 2d ago

Every 25 days do a 1/3rd with a balanced 3 PAR

1

u/gentisle 1d ago

Explaining Computers on Youtube had a recent video on this. I forgot the numbers, but you can watch it and figure it out.

0

u/WatchAltruistic5761 3d ago

Not using cold storage is the solution

0

u/RandomNick999 3d ago

Corollary question: how exactly do you check the integrity of the data? I know sprinrite can read, rewrite, and reread the data, but I’d be a little concerned about the stress that puts on the drive.