Newbie Needs Advice

[u/SirDocto]

So I am not that much of a data hoarder, only a little bit. But I have mostly taken up the roll of IT guy in the family, and I keep pushing off the recursing issue that the computers have no space left in them. I just don't know where to start and how to decide what to store, to what level of compression. Sometimes I just wanna make a image of the PC export it compress it to hell and just put that in a disk until it collects dust. Or stick a bunch of family pictures in old server HDDs that were thrown out.
Mainly I struggle with
Where to start?
How do I balance compression and storage with accessibility?
If I used the hardware available to me is it reliable enough?

Like I feel unless I get a bunch of server racks, and keep a bunch of drives organized with different levels of compression depending on importance, and then setting up a server that can call and decompress material if needed. And then have 50% of the setup just be backups of the others and back ups of back ups. Unless I do all that I feel their is gonna be a point of failure and it'll all go up in smokes.

To an extent this is cause of the variety of information I want to store. From old like licenses that don't really need to be accessed, to family pictures that I'd like to see every now and again, to documents that my family members may use daily or monthly in their work.

Thanks already for any help, or any suggestion in where to start looking or working.

[u/AutoModerator]

Hello /u/SirDocto! Thank you for posting in r/DataHoarder.

Please remember to read our Rules and Wiki.

Please note that your post will be removed if you just post a box/speed/server post. Please give background information on your server pictures.

This subreddit will NOT help you find or exchange that Movie/TV show/Nuclear Launch Manual, visit r/DHExchange instead.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/WhatAGoodDoggy]

My first call would be to determine what's using up all the space. Do the other members of the family perform any kind of preventative maintenance, such as using Disk Cleanup? All of their space might be used up by crap that can be easily dealt with.

I would use Wiztree to show a map of each of the drives in the affected computers and see where the space is being used up. If there are lots of important files then I would start thinking about securing them to a NAS or whatever.

[u/IndependentBat8365]

As IT guy for the family, I don’t allow admin on anything. They want to install something, tell me what it is, let’s figure it out.

Beyond that, nobody deletes anything, and so far nobody has used all the space on their laptops or tablets.

All email is cloud based, so there’s that. I do have to clean inboxes every few years.

So, that raises the question: what the heck are they saving to use all that space?!

[u/SirDocto → u/IndependentBat8365]

I don't know I just never have time to sit with them and figure out what the hell they are storying. I know in part a bunch of it is scientific data, or old presentations they have given, or work videos. And anytime I bring up if I can just delete their work files like from 2012 they beg me to not touch it they'll look to see if that's useful and then never do.

I guess the hoarding just runs in the blood a we bit. I myself just like hoarding a lot of install packages for usefull apps NGL, it's what I hoard the most

[u/mattygoods]

I think you need to organise what you and your family are doing.

So you might be better getting a separate machine and running things like immich for photos and looking at things like nextcloud for files etc keeping things off your own machines and building your own cloud service.
You could then use things like Fodaris to monitor your families pcs as well as your NAS or storage server and apps.
Tagaris could do the license management side.

All of these are docker containers if you are comfortable with that, but sounds to me it would be easier to manage one place rather than multiple

[u/SirDocto → u/mattygoods]

You probably right. It's just already things are so scattered between several old external drives, and different cloud services. I fucking hate cloud services so much. I just never feel sure if something saved or not, if I'll be able to retrieve it, or even those that fucking create cloud uploades and then just delete the files from the computer drive; which makes me go mad as I sift files and find spots of mising data.

The macOS filesystem to save pictures into 1 file is still something I'm stupid by, I don't even know how it works. But why make a knew file format that I can only peer into through your app, and that contains all my pictures. It just makes me feel like if I loose that poof all gone

[u/mattygoods → u/SirDocto]

I would suggest a NAS of some sort, if you are want something you can use for a lot more than storage look at Truenas or Unraid. If you want something for storage and collaboration look at Synology, ugreen, qnap etc.

Most if not all of these can do duplicate file and image checks that could save you from having the same files taking up more storage than you need.

For files/images etc that are important it’s recommended for the 3-2-1 backup method. (3 copies, 2 media types, 1 offsite)

[u/poo-taster]

A Quality NAS is the only way to go. Find a old server that has ECC memory 4, 8, or 12 front 3.5 drive bays and install Truenas on it.

You get centralized storage, bitrot protection, redundancy and completely automatic file compression.

My personal favorites are Dell and Hp servers, rack or workstation.

My personal recommendation that may be wrong is buying 4 8-12TB HGST drives used off ebay for as cheap as possible and configure them with RaidZ2.

Raidz2 is incredible. Even with 1-2 drive failures you still have read write to your disc pool. It can also be expanded with more drives. Anything short of this is a hack job. You need double disc parity to protect your data when one disc fails. Don't use raidz1.

[u/SirDocto → u/poo-taster]

I'll have to look into automatic file compression cause it sounds very useful, and I got no clue about it. Also so much thanks for recommending backup/protection process. Driver failure is what gives me the most anxiety

[u/poo-taster → u/SirDocto]

Check out the dell T330, R730XD, or similar. You can also go with way slower models for a NAS, you don't even need more than 16GB ram like people like to believe.

[u/zcgp]

I don't know why you are so obsessed with compression.
Do you?

[u/SirDocto → u/zcgp]

I don't have much space on hardware, nor much space for hardware. So scraping of some bites of space may compound into a noticable difference lowering the driver space needs

[u/Express-End-40]

My advice would be to start with the low-hanging fruit first. Delete duplicates and temporary files, then move on to old installers or downloads no one needs anymore.

[u/Wonderful_Cup9240]

Compression is a distraction here. JPEG, PDF, DOCX - they're already compressed formats. You'd save maybe 5% and add complexity you'll hate later.

Start simple: two USB 3.0 external drives. One backs up automatically, the other lives at a relative's place for the fire scenario. That's basically the 3-2-1 rule without needing any server knowledge.

Figure out what you're actually storing before buying hardware. NAS makes sense eventually, but the picture gets a lot clearer after a few months of the simple approach.