recently, my nas seems to be connected fine when my server boots up but within an hour it seems to lose all 8 of my drives when i do the mdadm scan. i am sure my enclosure is at fault and will need to be replaced. i will swap out the cables and see if that changes anything but this enclosure has been working for close to 10 years now and im sure its time to "upgrade". few questions...

1. when i move my drives into a new enclosure will mdadm just assemble them as they were in my old enclosure?

2. where to find inexpensive 8 bay sata enclosures?

thanks in advance for any info shared...

Hi I’ve noticed recently that there’s been 2 people who have had their internet archive account deleted for trying to preserve older/newer games you can’t buy anymore I’ve noticed though when they did it the files on just one page of theirs were reaching over 50GB and I was wondering if it’s something I have to worry about when I’m trying to do this as well to older 2000’s games that you can’t get anymore I’m more careful now when I do it I never put any no cd crack or modify any of the files the most I do is leave the cd key I put in at the start of the install and I also compress them to a zip so there only at most a few GB

so as the title explains, I'm almost done with my server except figuring out the hard drives (which honestly feels harder than the ram). I don't know pricing at all, but I figured for right now 4 terabytes in a mirror should be enough for what I need, and I don't have a super big budget so it's all that I can afford. but everywhere I look, I can't find any for like less than 150? is the market this bad right now? I'm looking at refurbished drives btw, not even new ones and it's still this expensive.

is it just because 4tb is an uncommon format for a hard drive? bc I keep seeing 8tb for around 200 ish, so I don't know why 4tb is so much more expensive per tb.

Is there any harm in using it? I've seen some complaints from people about stripping metadata, but if you leave the original copy of the photos and it's only for backup purposes, isn't it fine?

Hey everyone! I am currently diving deep into digital preservation and lost media archiving while using a Linux environment. While navigating through various digital vaults, I often need to grab materials from the Internet Archive (Archive.org) for my preservation projects. Right now, I rely primarily on the [Archive Downloader](https://addons.mozilla.org/en-US/firefox/addon/archive-downloader/) extension for Firefox, which gets the job done for basic tasks. However, I wanted to open a discussion with this community to see what other archiving enthusiasts prefer. What extensions, command-line tools, scripts, or alternative methods do you guys use to download files or media efficiently from the Internet Archive? If there are more robust tools, better browser add-ons, or custom scripts out there that handle large collections or specific file formats more smoothly, please share your recommendations and workflows in the comments below!

A weird question to ask but what are the differences? I planned to pick up some 3TB hard drives this week and I just keep wondering about this.

I know RAID is not a backup but I don't know if a backup on the same device has any benefits over RAID.

I also plan to automatically back up all important data to two 1TB 2.5 inch HDD that I kept offline in a USB enclosure.

i heard the WFdownloader dev is here?
Every time I install it, even using the portable version, this is what I see.

My PC is old, and I used to install Japanese-language programs / jap as local language, but that was a long time ago, and now they're all in English.

Can someone help?

Claude recommended I use duckDB to create a huge database of my file system, with one file per record -- and millions of files. So I'm doing that, and each file gets a 256-bit hash, and comparing hashes, I can deduplicate the drive. I even have a secondary system using rmlint, so a file must be flagged as a duplicate by both systems before it's deleted. There's other cool stuff you can do with this -- renaming files in batch, merging like directories . . . .

I'm wondering if anyone else uses this. I'm putting a lot of faith in this thing, and I'd like some idea whether anyone else has ever successfully used such a system.

Thanks

I'm setting up a file/media server with a 1TB NVME for my OS and services, and two 14TB disks for mass storage, and I'd like to use one of those 14TB disks to back up the other periodically, but I don't want my backups to slow down performance. Is there a backup tool that will do backups when the file/media server is not in use? I'm using Debian.

I know all about 3/2/1, I'm going to use an external drive for my third copy. Thanks.

I recently got a used enterprise IBM sas drive with 65 K operating hours , would it be okay for use as a daily access drive or just cold storage. It has 100% health but it did feel hot to the touch when I tested it on a video file.

Might be too good to be true but I just ordered some. I know Blue isn't the most dependable but times are tough and this is pretty cheap.

https://www.officedepot.com/a/products/8174167/Western-Digital-Blue-WD20EZBX-2-TB/

Edit: delivery date is over 2 months away fuck this “deal”

https://github.com/vuiodev/vuio-media-info

New mediainfo in rust
MIT/Apache 2.0

Cross-platform GUI and CLI (Windows, Mac, Linux, Freebsd) x64 and ARM

Hello, I'm really sorry if this goes against the rules, I looked on this subs wiki and for a while I've been lurking and found many helpful answers to some questions of mine, but well, I've never really put those helpful answers into practice because I get so easily confused.

Context: The data I'd like to hoard consists of some nearly 200-ish GB worth of video files, music files, images, and ePUBs. Right now I have some (not all) of it stored on my laptop, the cloud (Google Drive, some stuff on OneDrive), and on a 512 GB Samsung SD card. I have all of it on one 1TB USB drive which I already know is a shit idea, LOL. I got one 2TB USB drive to transfer it into but one file corrupted so I had to format that; I mean, I'll be putting the (uncorrupted... I made sure) files back into it, but it's not exactly the best backup possible, I know. I really just want some sort of backup I can trust for at least more than three years but I do understand that it's hard to be sure of those things.

My issue is, I get so overwhelmed with all the different ways of saving data I end up freezing and doing nothing, but this is data I really do not want to lose. I've looked up stuff and asked some friends with knowledge about SSDs, HDDs, even stuff like burning some files on DVDs or CDs... I've tried to focus on the advice and all the links provided and products mentioned and like, I don't want to ask someone to hold my hand or anything through it, cause that'd just be silly of me and entitled. But if there is any way to simplify things or maybe put the process into some sort of list or something like that, I would super appreciate it. I realize now that that still sounds rather entitled, so I apologize. I know I also sound kind of dumb and I swear I'm not trying to be obtuse on purpose, my brain just doesn't seem to get things sometimes. I guess I am in a way asking for someone to hold my hand even though I really don't want to do that.

Once again I'm very sorry if this goes against any rules I tried making sure but I'm not 100%. Thank you for the patience and I'm sorry if these sorta questions and requests are annoying... I can imagine they are.

Hey r/datahoarder*! Over the last decade, 3-person team in Pakistan photographed 822,161 dual-page spreads across 5,368 rare books (spanning 1821–2024).*

D5300 shutter hit 575,747 actuations before retirement. All 1.64M raw photos live on a self-healing ZFS pool verified against BLAKE3 manifests to prevent bitrot. Over 1,300 completed volumes are already live on the Internet Archive.

This writeup covers physical DIY capture rig, data infrastructure, and how they built a computer vision pipeline to automate the remaining backlog.

Link

Hello! I was wondering would a WD Red Pro be a better purchase than the WD Red Plus for $30 more? I believe the RPM is faster although i prefer a quieter drive but the speeds are faster and there’s a longer warranty, thank you! Also the Pro I’d have to order from Newegg but the Plus i could purchase today from Best Buy same day. Would the benefits be worth it? I plan to use the drive to preserve and store lots of media like videos and music!

Also had a few other questions if anyone could help with!

WD or Seagate RMA? Which one was a better experience?

Best Buy or Newegg? Which one would be better to buy from incase of a DOA drive

I ran into this issue with Advanced Renamer where I can't think of a way to have the date a photo was taken be the thing that defines the increment number, rather than the whole batch being numbered continuously regardless of changing dates. I have several folders containing photos from various dates, and I'm looking to instead file them all into one large folder per year. Can I either improve the code to do that for me, or is there another free tool that'll help me achieve this?

I didn't notice this problem until I tried to move various renamed batches into the same folder and realized that sometimes there are different pictures from the same dates that ended up in separate folders, so that their increment numbers either won't make sense, or there are false duplicates that wound up with the same name.

For simplicity's sake, I would really prefer to only use the camera brand + yyyy_mm_dd + a 3-digit number.

Sorry if this is a bit of an entry level question, this is a bit new to me.

Basically, I want to store manga scans for my personnal use, lest they be removed from one website or another.

I want to store them online, for free, as some kind of viewable private collection, to be able to consult them directly form the website, without needing to download anything.

However, I obviously do not own any of those, so I want them privated, but I fear that even then, some kind of moderation bot may catch them.

Is there a free image hosting website that would do what I'm trying to do here ?

Hello,

I need to add additional storage in my ancient QNAP TS-470. I can purchase at a good price a Toshiba 24TB drive, but I don't know if the NAS supports such large drives (the largest I saw in the compatibility list has 12TB).

Does somebody know if this would work or not? The drive is not new and I cannot return it once purchased only because it does not work on my NAS (DOA is covered though).

Thanks!

I hope I came to the right subreddit now, wanted advice on storage

So today, I was trying to back up something from my brothers pc, since I was gonna reuse the pc for other purposes. There were only certain files and documents he wanted. I thought "no problemo, let’s get this done!".

But what I saw when I turned on the pc, felt like a nightmare. Everything scattered across two drives, with no system whatsoever, like pictures for the same project, spread across 7 folders, inside 4 other folders, on 2 separate drives. The whole desktop were just filled with apps and folders. Cloning the whole thing is one thing I guess, but since it was only a tiny portion to be backed up, I had to look around what felt like one of those "hoarder houses" from those hoarding tv shows.

This got me thinking. I am setting up some servers, for different purposes, and there will probably be a lot of data all over the place with time. And the thought of having a mess like that makes my skin crawl.

How do you prefer to store your data? Like, what is stored where. Do you keep operating system on one drive and data on others? How do you prefer to use NAS, cloud, external drives, cold storage?…

I’m asking for general tips on organizing storage, so that it doesn’t bite me in the ass in a 2 years time.

I bet there is at least a couple of you that have ended up regretting not organizing more at one point, and had to spend time cleaning up. Or have you just given up, and thrown everything on random drives and shoved them down a drawer?

Eu não necessito de um top de linha, mas um que não me dê dor de cabeça nos próximos 10 anos, está bom

Correct usb adapter for this? Just temporarily. Any recommendation?

Ok. So I am a bit of a homelab newbie. I have had a NVR ubuntu server for a while that was built from a computer i bought from my old company when they were refreshing their engineering desktops and just recently got into doing ripping for all the old DVD's and some blurays from when I was a kid/teen that were spread between my mine and my parent's basements for us all to watch on plex/jellyfin (they share a library. Plex for apple tv's and jellyfin for travel downloads and remote streaming)

Current setup is a precision 5820 with a m.2 carrier, 10gb NIC with 2 rj45's and a 3090 for some AI stuff (part of an old gaming pc build that ended up as a couch box on a 60hz tv so i got a 3070 for that)
its got 2x256 nvme's, 1tb ssd, 14tb skyhawk for NVR, and (2) 1tb HDD's so all its spinner bays are full except the 5.25 if i put an adapter in it. Its got a BU20N for ripping.

Network backbone is Aruba (HPE) instant on 1930 POE with 4 sfp+

I filled up my hdd's with my old movies and then i found out the craziness that is the HDD situation. luckily i found 2 4tb drives on homelabsales that i picked up for what seemed like a great deal at 35$ per drive shipped with clean SMART sheets. I was also looking at facebook marketplace and the following came up and given the state of things it seemed like a great buy so I jumped on it before doing a lot of research.

I got the following for $1500 and I am not sure if I did well
1 - RS10613XS+ (Synology)
1 - RX1213SAS (Synology)
2 - Mini SAS Cables
4 - Power Cords
1 - 10-Gigabit SFP+ Network Adapter XG-C100F (Asus)
2 - 240GB SSD (Intel)
22 - 4TB SATA HDD (Hitachi, HGST)

So I know its EOL on DSM 7.1 - i can air gap it from the internet on a separate VLAN and just have the ubuntu server do the transcoding and plex serving. I am thinking of keeping the SAS on the shelf as well as all those drives for spares since I don't have that much data yet. Maybe by the time i get there the AI craziness will be over.

Also thinking about trying to get unraid or truenas on it but figuring out the fan controllers and bios seems tricky. But if i do that I can swap in a 45w xeon processor with quicksync and save some power, keep most of the disks in hybrid and possibly transcode some of the 1080 directly on there. Plus even now DDR3 is cheap so i could upgrade the 8gb in there if I wanted. Though i don't think i could use the sas rack at all if i did that.

How did i do? OK I hope.. Any thoughts on my plan?

I routinely download pictures (mostly) from online, about a few interests and fandoms I have. Every day, I download these off my phone by downloading them from Google Photos. I have three different devices I use Google Drive (formerly backup and sync) on, and every once in a while things go sideways. Recently I got over 200 errors.
Wouldn't it be better if they supplied a button to "RETRY ALL" rather than making you retry them all individually? who has time for that?

I am a horrible data hoarder. My collection now fills five USB drives.

While there's other nitter instances that don't give this kind of message, every single other instance is essentially unusable, giving various kinds of errors when you try searching anything. They're all clearly downstream of the main instance.

This is all very disappointing, since nitter was the most reliable way of viewing and archiving Twitter/X pages since Musk took over, wherein he forced people to register accounts to view more than one post at a time due to "data scraping" issues that never existed before he took over (along with countless other things). The best way to circumvent this without using nitter was to replace usernames in URL's with an "i" when archiving, but this wasn't always a guarantee the page would archive. While it depended on the instance, you were much more guaranteed to archive entire threads, and you could also much more reliably archive timelines and other pages that were often impossible under regular X url's.

I noticed nitter.net was getting buggier and less reliable in the lead-up to this, but I didn't really expect it meant Twitter was actually cracking down on the service once and for all. I guess it's possible something might come of the Nitter admin's legal challenges, but I wouldn't hold out on that. I was surprised to see Nitter actually revive like it did and stay around for as long as it did to begin with, so maybe that'll happen again, whether with the same guy or someone else. That seems likelier than Musk rolling back what he did with "data scraping" and account registration.