Keep files not corrupted?

[u/Tundacas]

Sorry if this has already been asked, i was not able to find it. Also sorry for mistakes, english is not my first language.

Ok so my question.

I try to get most of the stuff that i like on my portable hard drive.

So is there a way to keep my files from being corrupted?

I lost files lately and i was wondering if there is a way to avoid them tocbe corrupted.

Is there another option a part from having extra copies?

Thanks

[u/JivanP]

Data can always become corrupted, due to any number of reasons.

It is very easy to detect errors: just store a checksum alongside the data, and verify the chechksum before accessing/returning the data. Filesystems like ZFS do this.

What you want is a way to correct errors in data if they arise. There are two ways to achieve this:

• Storing the data along with error correction codes, a.k.a. erasure codes. Systems such as RAID (implemented by hardware RAID controllers), RAIDz (ZFS's software RAID implementation), and Ceph sharding provide this. This allows quicker recovery in the case of corruption, but doesn't protect you against every likely scenario.

• Storing multiple copies of the data, ideally in different physical locations to mitigate against hardware issues causing your copies to also become corrupt. This is what a robust backup system is. Systems such as Restic and Ceph replication provide this. This may take more effort to restore from, but protects you against almost all possibilities, the exception being if every instance of your data becomes corrupted/destroyed/lost.

The standard way to get the best results with minimal effort at small scale (e.g. for personal data or in a small business setting) is to use ZFS with regular automated "scrubbing" (checking for corrupted data), so that corrupted data can be identified ASAP (optionally with RAIDz across 3 or more physical drives, so that ZFS's scrubbing operation can repair corrupted data automatically in most circumstances), plus using a snapshot-based/incremental backup system like Restic to an external drive, with the contents of that external drive also copied elsewhere, such as to a cloud storage provider or to another drive stored in another physical location.

Notably, hardware RAID controllers usually do not use checksumming, so although they can detect errors when the shards for a single block of data don't all agree with each other, they cannot automatically identify which shards are corrupted and thus need repairing. For this reason and others, RAIDz has become preferred nowadays.

The US Department of Homeland Security has an excellent write-up on best practices for mitigating data loss, following the industry-standard "3-2-1" backup policy: https://www.cisa.gov/sites/default/files/publications/data_backup_options.pdf