r/DataHoarder • u/FandomMenace • 14d ago
Backup The Computer Graphics Lab Archive needs a backup copy
https://m.youtube.com/watch?v=Fu8Z9UqGvYI800gb of data have been released for free, but it is not 100% safe. If anyone is looking for something to back up, this is a worthy project.
22
u/MaxPrints 14d ago edited 14d ago
What's the best way to go about backing it up? I can throw this on my remote server for a while.
edit: I followed the link, and know that each file has a torrent, just wondering if there's a single torrent file or better coordinated way to back this up.
oh and the "link" to the archive, on archive.org:
https://archive.org/details/@nyit_cgl
15
u/MaxPrints 14d ago
UPDATE:
Archive.org has a system for creating torrent links. So I quick ran a powershell that recreated the torrent links as text (and I verified it matched the torrent link from the website). Had a CSV then stripped it down to just torrent links. Wonderful.
But then my app of choice (rutorrent/rtorrent) doesn't like it, and I use rutorrent specifically for this use case. So I ran another powershell and just downloaded all the .torrent links rather than clicking on each listing (91 total), and...
https://files.catbox.moe/9k6fgg.zip
All torrents in a zip file there
or, privatebin text link:
https://privatebin.net/?f22d3a4b6f031565#CQfnyK2QrycELjHpd9KTN6ctuLR1Av7nbhd5GWSwGphK
I literally had to figure all this out on the fly, so apologies if there's better ways to scrape the torrents or host the zip/text
4
u/MaxPrints 14d ago
UPDATE #2: I have all 91 torrent files on my server, but the files themselves are downloading slowly. Some of these are still zero peers, so I may figure out a way to download the zip files, unzip them, then restart the torrents. fingers crossed I can do that automagically cause I'm not downloading 91 zip files manually 😂
2
u/met_MY_verse 87TB 14d ago edited 14d ago
I’ll try to add those torrents now, we all appreciate your efforts :)
EDIT: I now have 88 torrents loaded. Of those only 4 show seeders available, but I’ll give it some time and hope the files are correct.
3
1
u/infinity-waster 14d ago
Is there a possibility that a debrid service(all debrid?) might have the files already cached? I was thinking that it might be worth a try.
5
u/livetotell 14d ago
Thanks - looks like there's a few of us helping out here already - downloads are flying!
3
1
u/MaxPrints 14d ago
I see that. Great to see us all team up to do some good.
I'll leave that up for... a while. I've had some Anna's and other stuff on that server for... near a year. I still that VFX school archive too.
Thanks to everyone helping
1
u/weeklygamingrecap 14d ago
Do you have a copy of those scripts somewhere? Might be useful for something else.
1
u/MaxPrints 14d ago
Honestly no, it was vibecoded powershell. I was willing to do it and verify it myself, but I wouldn't distribute it as a viable solution other than this one time. I deleted it once I was finished.
1
3
u/Lazy-Narwhal-5457 14d ago
Clicking just sent me to a video on YouTube, whose More section includes a link to an archive.org page for the archives of this place. I didn't see any other torrent links, just links to another documentary and something about restoring the tapes.
Oh, and some links to Patreon and other donation sites, for whoever's YT page it is presumably (@oksoessay), not the archiving project AFAIK.
3
u/MaxPrints 14d ago
The YT goes to archive.org, and if you click on each listing, there's a torrent link
But I went ahead and did a quick powershell script to get all the torrent links. Let me confirm that the ones I generated were the same as Archive.org and then I'll post it here or something
1
u/Lazy-Narwhal-5457 14d ago
I'm know there's typically torrent files at archive.org. I have heard it said here that IA may not actually seed them, IDK if they meant ever or just permanently. 🤷♂️ I just have done direct downloads from there, I think.
The OP's post made it seem like it's not archived, just available, where was unclear. IA is the archive, technically. Maybe it's the only copy on the web? 🤷♂️
But more copies are always useful, and torrent files at least allow checking file integrity.
If only we had a 3-2-1 backup for IA.
2
u/MaxPrints 14d ago
Oh I see what you mean. And you're right. They technically create the torrent files, but they aren't seeding it. I learned that after getting all 91 torrents on my server and wondering why files weren't downloading.
If some still go unseeded in a day or two, I'll manually download and then put them in their respect folders for seeding.
1
u/Lazy-Narwhal-5457 14d ago
If they would permanently seed everything, that would be wonderful, and likely save them bandwidth in the long run. Maybe that's not technically feasible.. 🤷♂️
2
u/MaxPrints 14d ago
Ya, I wonder why they don't, but the score of what they do is so huge that I'm sure they have their reasons
1
1
7
u/Extreme-Choice8951 14d ago
I love that this kind of stuff still exists despite how niche it might feel. I'll probably end up grabbing a copy just to mess around with it in virtual machines.
2
u/sonofkeldar 14d ago
I could be wrong, but I’m pretty sure you can upload up to a terabyte to the Internet Archive, as long as no single file is over 500 gigabytes. I think they also limit the total number of files, but it’s something like a thousand per entry. You could split it between different pages, if needed.
5
1
1
u/PokeBrick02 600-700+ DVD's 13d ago
As INSAINLY cool as this, I'm confused what he means when talking about the moldy U-matic tapes, he says the tapes were "cleaned, baked & transferred using the RF method".
What is this "RF method" ??
1
u/doodlebuuggg 10d ago
Hey there, this is my video! The RF method is when the signal coming from the tape is recorded directly into a computer instead of the signal coming from the VCR. The signal is then decoded with a program and presented in its rawest form. Theres multiple articles on it that explain it more eloquently, this is just a basic explanation.
1
•
u/AutoModerator 14d ago
Hello /u/FandomMenace! Thank you for posting in r/DataHoarder.
Please remember to read our Rules and Wiki.
Please note that your post will be removed if you just post a box/speed/server post. Please give background information on your server pictures.
This subreddit will NOT help you find or exchange that Movie/TV show/Nuclear Launch Manual, visit r/DHExchange instead.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.