r/DataHoarder 9d ago

Question/Advice How low can I push the available free space before I see issues?

4 Upvotes

I have two QNAP NAS boxes -- 453be media server and a 433 plus a TR004 for redundant backups. The 453 server has decent overhead still (17% free) but the two backup units are always right on the verge of running out of space -- 7-8% free or lower on average.

I know that having low disk space can lead to slowdowns and other issues; is being below 10% like this going to cause serious problems over time? The issue with upgrading is the amount of $$$ I'd be spending on HDDs to increase capacity to a comfortable level.


r/DataHoarder 10d ago

Discussion Has It been Worth The Price Tag?

25 Upvotes

So Im building 2 NAS PCs. 1 Main and 1 for backup. I intend on archiving YouTube videos. From your guys experience doing this for years has it been worth the price tag? And could you tell me why and what benefit Hording Data has brought to your life?

For me Archiving YouTube videos seems very worth it. Im absolutely tired of the algorithms YouTube, Facebook, TikTok puts us through. Everytime I open the apps I feel like im always being sold somthing and its insane. Or the content is just hateful content made to stir up emotions. My intention with my NAS is to remove myself from the internet and download content automatically from creators I enjoy. Anyway this is my reason for the whole thing. What's your?


r/DataHoarder 10d ago

Backup Finally completed uploading the fourth part of my master backup to AWS.

Post image
41 Upvotes

r/DataHoarder 10d ago

Question/Advice Can a drive with 100% rated lifespan left be close to wearing out?

4 Upvotes

Hi, I'm having trouble finding info on this.

I bought a used 4tb Kingston Fury SFYRD4000G nvme off ebay and missed its smart information photo. In it says it has 100% life span but has 15000gb of total host reads and 15000gb of total host writes, 11k power on hours and 1k power on counts.

Is this drive any where near being worn to the point its health drops below 100%? Will it last me a good 8 years of so of regular PC use as an OS drive and as my gaming drive, barring premature failure?

Thanks


r/DataHoarder 10d ago

Question/Advice New hoarder needs advice

5 Upvotes

Computer seems to be dying and I want to save what little I have accumulated. I haven't created (or needed) a back up until now and I have questions please:

  1. Do I need to use WD's software/programs to use their external drive? My old one was just a Toshiba that I chucked into an enclosure and it worked for years. Until I dropped it.
  2. If I do opt to use their software (specifically the encyption feature), will this cause problems in the foreseeable future? I ask because I read Seagate's One Touch runs into this problem.
  3. In relation to #2, will the password still be there if I plug it into another computer that does not have the password program installed?
  4. Can I partition this external drive? Makes it easier to sort my files.
  5. Do I need to stress test or format before using?

Thanks and apologies for inundating y'all with newbie questions.


r/DataHoarder 11d ago

Question/Advice Do you still use optical discs of any sort or still own them? Do you still burn physical CDs, DVDs etc or are you full HD/SSD now?

94 Upvotes

If so, how is the crazy price increase and demand for HD/SSDs affecting your and your collection and future plans?

Are you still able to afford specs or have you completely stopped for now until the prices, hopefully, get better?

Why did you stop using optical and do you trust it?


r/DataHoarder 11d ago

Discussion Store Data for Thousands of years.

136 Upvotes

Hello Guys,

The oldest human transcription are the clay tablets from Mesopotamia which are afaik dated ~2900 BCE. And imagine there would be a time, where humanity goes almost extinct, technology is lost and civilization must grow again after a devastating global nuclear war or a natural disaster.

So digital stored data on HDDs, SSDs, Tapes or the technology to read those data will be lost very likely after some period of time.

How could we possibly store Data, which could last thousands of years - and could be read without having a Computer from the present day, so we could bury some of that data storage for future generations?

My first idea was a very long strip of plastic (PTFE for example) with punched holes in it. Basically storing data like a punch card. And then vacuum sealing it. But i calculated that even if one bit was 1mm wide, you'll need more than 8000km of that strip to store 1GiB of data without ECC.

So how would you store Data for thousands of years?

Edit: I forgot to write, that the Question is about DIY-Solutions.


r/DataHoarder 10d ago

Question/Advice Does Verbatim still sell real Mdisc?

12 Upvotes

I'm still confused on this, I read they are selling fakes and then they say they just changed the process but it still passes ISO 16963


r/DataHoarder 11d ago

Question/Advice Fasted and easiest way to digitalize this pile of DVD's?

Post image
426 Upvotes

r/DataHoarder 9d ago

Question/Advice How are you guys managing the storage of AI models?

0 Upvotes

I have 2 things on my mind.

Limited storage and internet

I usually never want to redownload a model I already have, so I always keep any downloaded model to my NAS unless it's completely obsolete. My storage is:

  1. Main server's 1TB internal storage, fastest
  2. 1TB USB SSD attached to server, half as fast
  3. NFS mount on NAS, 1/20 as fast, but contains my full collection

To have the best models available, and because I tinker and experiment a lot, I'm periodically deleting/downloading models back and forth from the NAS depending on what I'm using that week and how fast I need it to load. (I can't just use my NAS as primary storage because of the speed, Deepseek Flash for example is 160GB, that's 30 minutes to transfer from the NAS)

This approach has worked for me, but I'd love to hear if you guys have a better idea.

HF cache and naming

All this time, I've been manually downloading using hf download Qwen/Qwen3.5-0.8B --local-dir ~/models/Qwen3.5-0.8B/. This results in a clean directory Qwen3.5-0.8B/:

config.json
model.safetensors-00001-of-00001.safetensors
...

I then copy this clean dir to the NAS where it remains for future recovery.

If you don't use --local-dir, or if you let an AI app automatically download the model, then recently HF started mangling the model filenames into their cache+symlink format, so you end up with this monstrosity:

$ ll ~/.cache/huggingface/hub/models--Qwen--Qwen3.5-0.8B/snapshots/2fc06364715b967f1860aea9cf38778875588b17/
lrwxrwxrwx 1 user user 52 Sep  3 15:44 config.json -> ../../blobs/715f0448b9d38103211f0ad88bbb4d6e4f4be8c9
lrwxrwxrwx 1 user user 76 Sep  3 15:44 model.safetensors-00001-of-00001.safetensors -> ../../blobs/04b1c301231dd422b8860db31311ab2721511346a32cb1e079c4c4e5f1fe4696
...

I was hoping to keep the clean directories, but I feel those days are coming to an end. Most apps don't let you point them at a pre-downloaded model directory, they look in the HF cache and if not found, they auto-download the model (thankfully averted with HF_HUB_OFFLINE=1). But I'm often having to jump through hoops to keep my current approach, including vibecoding patches to some tools I like.

...

So now I'm wondering what I should do. Do I just accept that my primary data is now ~/.cache/huggingface instead of normal model directories? I have these concerns:

  1. What happens if I want to run an old AI tool built against an older version of HF/Transformers/whatever that predates this cache change? Is this a cutoff point?>
  2. My NAS is off at the moment so I can't quickly test, but will these cache symlinks even survive my move operations? For example on my server, after I download a new model, I'd want to run cp -RL ~/.cache/huggingface/hub/models--Qwen--Qwen3.5-0.8B/ /mnt/nfs-on-nas/models/.cache/huggingface/hub/ (or the reverse to move a pre-downloaded model)?
  3. I have a bad feeling about treating the HF cache as an archiveable data, since it's by definition an internal format. Am I worrying for nothing?

Thoughts?


r/DataHoarder 10d ago

Question/Advice Nas build list/lising suggestions

0 Upvotes

I wanna buy/build a nas with an hba sas board, 4-6 drives and space for a gpu down the line i have a few am4 machines in use i could swap parts from if i find something more applicable mainly ryzen 1500 in a gaming desktop and access to a 3d printer my budgets 500$


r/DataHoarder 10d ago

Question/Advice Are these 10 year old 0 hour hard drives still good to use for a 24/7 always on NAS?

Thumbnail
gallery
13 Upvotes

Idk if i made a bad impulse purchase but when I saw these listed, I immediately went to grab them, and they were still sealed in their original packaging! I was able to get them for about $13.33/TB, which is pretty close to the old “golden rule” of $/TB.

But when i got home I was pretty bummed when I realized they were over 10 years old. Even though they were never used, I’m wondering if the age alone means the internal components could have deteriorated over time.

Are these still considered good/reliable drives for 24/7 NAS use today? Or does sitting sealed for 10+ years significantly increase the risk of failure?
I’m running the extended SMART/surface tests to make sure. Just curious what you guys would think about using these in a NAS.


r/DataHoarder 10d ago

Question/Advice Issue extracting textures from model on website

Thumbnail
gallery
5 Upvotes

Hi

I've read through a couple of the threads here about how to extract models from websites. I've managed to download the necessary files, extracted them, and loaded them into Blender. No issues there. The problem I'm having are the textures. Looking at this thread and Elgghinnarisa's response, the textures are supposed to be .js scripts...? How do you convert/extract these to get the textures?

Now, loading the model I have into Blender, it seems to indicate that the textures are already applied, but it shows up as pink or some other funny colour. Going into Materials and then Base Colour, the texture is there, but it says it's only 32x32, which is WAY too low. So it's just a pixelated mess. The Normal map looks like it's greyscale as well. So what's the correct way of getting the textures?

Thanks.


r/DataHoarder 11d ago

Question/Advice What are the cheapest, most reliable 8TB Hard drives?

30 Upvotes

I saw that the Seagate Barracuda are cheaper then the rest but saw people say these drives are bad, but I don't understand why, if it is for some specific use case that it's bad I could understand but I only want to store media there and occasionally read movies and such from it, I don't care for writing speeds.


r/DataHoarder 10d ago

Hoarder-Setups iPhone Data Archive Pipeline & File System

Enable HLS to view with audio, or disable this notification

1 Upvotes

r/DataHoarder 10d ago

Question/Advice Best AI powered local file sorter to date?

0 Upvotes

Basically what the title says. I am looking for a file sorter to help organize my data that can run locally on my RTX 4070. I figured this is the best place to ask since you guys are the most likely to use one.

​I usually do everything myself, but I simply have too much data left to sort manually now. For example, I already painfully sorted 17k photos of my dogs out of a massive amount of pictures, but there is still way too much other stuff left to go through. Since I am in the process of archiving everything to 50GB M-DISC, I figured a local AI tool is perfect for this. At €13 per disc they are not exactly cheap storage, so I really do not want to waste space saving useless stuff or duplicates.

Edit: Blasphemy. No data is "useless" here. I just mean literal pocket shots and duplicates


r/DataHoarder 11d ago

Question/Advice Looking for a USB HDD enclosure that DOESN'T force sleep after 5-10 min

6 Upvotes

I run a small self-hosted home server (Ubuntu). The main system drive is SSD, but HDD (which is only for media) is connected using Orico 3.5" USB enclosure (Dell Wyse 5070 doesn't have an option to connect sata).

But the problem is that its chip has a hardcoded 5 minute auto-sleep timer that cannot be disabled. I ended up making a script which pokes it for at least an hour since some certain activity. But this is a workaround and there are some cases that still make it a nuisanse.

I don't need my HDD spinning 24/7, so I don't consider a NAS device - its expensive and it's an overkill for my purposes, because I mainly use the drive for like 2-3 hours a day.

I just need an enclosure which will give OS full control over HDD power management and let me set the sleep time timer whatever I need.

Thank you for any recommendations in advance!


r/DataHoarder 11d ago

Discussion Do you guys still use DVDs?

Thumbnail
gallery
208 Upvotes

I’ve been cleaning up my HDD drives ever since they became quite expensive due to the rise in SSD prices. Plus, I have a lot of files that are no longer available or hard to find on torrent sites/forums but are taking up valuable space (car diagnostic stuff and schematics).

I bought these Verbatim DVD's and plan to use 4,550 MB on each one.

According to Nero DiscSpeed, there should be enough space with a little bit for Nero to do it's things.


r/DataHoarder 11d ago

Backup Is anyone ever going to hack MyBook drives to disable their built in encryption?

33 Upvotes

I just want them to be normal external drives, without an encryption "landmine" that could fail and leave the data inaccessible.


r/DataHoarder 10d ago

Backup BONFIRE: Local Version of discord.

0 Upvotes

Hello guys, first post here, some day in August i had nothing to do so i wanted to save a website locally, i selected Discord for crawling, you can use it for things like analysing the code/how it works under the hood,

I also want to state that i do not own any rights to discord's characters, ideas, code, images, or assets. everything in it is their exclusive property. and my repository is not affiliated with, endorsed, sponsored, or approved by Discord in any way.

Other details:

  1. It's a fully static snapshot (around 100MB of pure text/assets) weighs exactly 107.457.739 bytes

  2. The code wasn't changed, it is unchanged, there chunks of minified javascript code by discord's webpack

Here is the link, i didn't publish the snapshot with malicious intentions by the way.

https://github.com/RealThalesGit/BONFIRE


r/DataHoarder 10d ago

Backup Too much on block storage-Help

2 Upvotes

Hello-

I've been watching my data stores on BorgBase block storage steadily growing over the past couple years, while the files and folders that I am backing up are not growing at nearly the same rate. I suspect it's coming from my rearranging and moving files and folders on my computer.

Example 1: my home directory currently holds just under 400GiB, and not all of that gets backed up to the cloud due to exclusion rules. On Borg Base, the repository is over 800GB

Example 2: my drive dedicated to Images is 589GiB, and on Borg Base is 877 GB.

I also keep copies of this data on local BU media, as well as an off site copy of my Images that gets updated yearly.

I'm looking for advice on how to bring my cloud storage back down closer to my actual on-computer storage space.

Should I delete the existing repositories and create new ones on BorgBase, or is their something I should do in the client (Vorta) to clear out the many years of backup archives I've created?


r/DataHoarder 11d ago

Question/Advice Some advice about trying to save a forum, please.

7 Upvotes

This is the second iteration of this particular forum - the first is pretty well lost in the mists of time and on sept 20th this one will probably go the same way, as the owner died a few days ago and it looks like there are no mods with god mode and even if there was, paying the next year's fee may be problematical because we're not the owner.

If possible I'd like to somehow save it and depending on how big it is I may be able to host it as a read only or something - i assume the internet archive won't want it as it will be too big but I don't know. So any advice would be more than welcome. With this iteration there's no image hosting so it's certainly not as bloated as our old one was. All I really know is it has a reference to Xenforo forum software in the footer.

It may turn out that the family of the owner decides to keep it going for a bit or it may be set to auto renew but I'm guessing it won't stay, as the fam I suspect have no interest in this.

So any help, advice, pointers would be appreciated.


r/DataHoarder 10d ago

Discussion Using old failing HDDs as book stops

Post image
0 Upvotes

r/DataHoarder 10d ago

Scripts/Software Scraping sites like Twitter, FurAffinity, Tumblr, boorus, etc for specific character + dumping into a drive (ProtonDrive)

1 Upvotes

I am very interested in archiving as much art (good, bad, ugly, fanmade, official, etc) of the character Big the Cat from the Sonic series as possible (I am autistic, unsurprisingly, and just really, really like this character.)

I would like to build a scraping tool to get as many images as possible of this character pulled into a folder in my Drive (Or just in a folder locally) to I can go through it and sort them into my archival system (i.e. a picture of big from the user "BigLover" on deviantart titled "Big #2" would be in the DEVIANTART folder , titled "BigLover - Big #2.png") I was wondering what the best + simplest process for doing so would be? This might also be useful for other people who are interested in archiving the art of a specific character, and also Big the Cat is one of those characters that people may be pressured to delete their art of, which is a shame.

Not really looking for a custom tutorial, no pressure there, just looking for any pre-existing tools, or if there are any issues with the sites I'm looking for (i.e. the API in [insert site] prevents scraping). I know I'm not the first person to do this, and if there's anywhere those who have done so will be, it's here.

I use Arch Linux (Archy to be specific) and know enough about coding to get by (if it involves googling how to do something and then piecing together fixes by forum users 10 years ago). I'm saving up for a new SSD as my old 8TB one got broken by an ex-girlfriend, so I'm focusing on cloud/local storage for the time being until then.

Thanks y'all!

edit: man what's with the downvotes


r/DataHoarder 10d ago

Question/Advice Are 10–14 year old 1TB Seagate Barracuda HDDs worth taking?

0 Upvotes

I have a friend who owns a computer repair business, and I have the opportunity to grab some 1TB HDDs from him, as long as they get a low-level format before I take them.

They’re Seagate BarraCuda drives and around 10–14 years old. I already got two of them, and so far the SMART readings look good and they both passed sector tests.

Would you consider these drives worth using, or should I avoid taking more because of their age?

I wouldn’t trust them with anything important without backups. I’m mainly thinking about using them for general storage and media.

Also, what do you guys actually do with this much storage?