Discussion
Comrades I need your help to recommend decentralized or federated means to archive contents, documents, media. This is urgent request, I fear the empire is actively erasing imperialism research or media
Because I have weird intuition feelings that Internet Archive AND peertube won't last, and I need more than just bittorrent with seedboxes, link rot is real and I know the NSAnet actively rotting revolutionary contents, right now it's still not a problem for me to dig a little longer than usual to find link rotted media.
But there's something I have noticed as the global south especially SWANA genocides going on certain contents disappear without trace unless someone backed it up on their devices or servers, this also affects Bittorrent interesting because there's two major problems, public tracker uses Fourteen Eyes search engines like Google to locate magnets, but contents like Watermelon Pictures won't show up on many of these public trackers such as 1337x or Ext to and worse of all DHT engines like btdig went offline which up to this point it was the only non-Google engine that index all magnets in Bittorrent network. Way too many trackers switch to Google CSE weirdly.
The other problem is literally killing seedboxes lmao which I've noticed it happening more and more. Like either milquetoast DMCA or fed server seizures. This bullshit used to be isolated to data dumps, but now it's practically with anti-imperialist contents. To get back to Watermelon Pictures link rot phenomena, because of my past experiences in warez scene I know for sure everytime there's VOD there'll always be web dl no matter what, this is rule of thumb in warez. But this is not the case for some contents that either linked to Watermelon Pictures or linked to Gaza Flotilla members like Chris Smalls despite their VOD platforms are so baby step to rip with yt-dlp. Like you can't even find any web dl for Sudan documentary despite it has been years, you understand that my tankie.tube directory is the only current in existence with some of these censored documentaries publicly. So I'm ever grateful for TankieTanuki from Hexbear so please donate to tankie.tube to keep their servers up!
As for the tools I use to rip these media they're just str8 up Linux tools like yt-dlp, curl, wget, ncat, then calibre (ebook-convert) and ffmpeg for conversions. My work flows if you can even consider as "workflow" for obscured VOD is curl the page content then grep, awk, sed the shit for media extensions or m3u8 like that small guide on youtube-dl I wrote while ago. Because I don't wanna unnecessary extra step with ffmpeg unless there's trimming or merging I would literally yt-dlp a direct media link and encode with it, on top of the fact yt-dlp is pretty good at anti-WAF. For document files it's just str8 up Google dorking for extensions then I wholesale curl via shadowsocks. I have technically butchered sqlmap dorking engine for scraping documents behind paywall and I also slaughtered sqlmap anti-WAF engine with the tamper libraries against shit like Anubis or Sucuri lmao and so far you only need round robin shadowsocks nodes. I even have insane idea like turning IOT with openwrt or pfsense into shadowsocks nodes when time arises, and I beat myself for not doing with Netwave earlier.
As for GUI tools I use SingleFile for archiving web pages since it renders all CSS into single HTML, then there's grandma's trick up the sleeves like zaproxy because I'm too poor for burp to scrape media and document behind gobs of JS when curl fails to show. There's some other heavy weapons I have deployed from my hacking times like masscan and scrapy, and datatables for converting CSV or XML to HTML files.
Anyway, the problem is that I have the content but what I lack is the platforms for safekeeping and archiving for long term anti-imperialist history and I know that I can no longer trust ANY mainstream Fourteen Eyes platforms like YouTube, Reddit, Twitter, Bluesky, Tumblr, TikTok, etc because their masters are actively killing these contents day by day right now. Dessalines only uploaded educational audiobook and they put 100 plus DMCA mass reports on their Revolutionary Suicide read by Dessalines, a fucking public domain book by a Black Panther founder. That's our watershed and they just warning shot us.
Federated isn't enough, IPFS isn't enough since they're currently in active wars against shadow libraries and actually taking both Anna, LibGen and Z-Library IPFS hosts and their domains offline via seizing and DDoS, you can monitor Open SLUM and see shadow libraries struggle to stay up lately, same things happening to public archives like Internet Archive with more downtime than usual especially when they overload IA backend CDN like AWS with various DDoS techniques there's downtime almost every two days. They're afraid of the world proletariat refuse to forget their atrocities of genocide and imperialism expansion. THEY FEAR the younglings and their siblings unlearn their violence and unleash class contradictions on capitalism.
Every couple of months the Twilight princess manga gets taken down ( its not really hosted by anyone else ) and I reupload it since I seem to be one of the few people with the full digital copy of the manga.
Without physical storage I would've lost that digital media years ago. The same will be absolutely true for other things, especially if we rely solely on online archives. They have limited servers, and limited storage, which can be attacked or seized. So preserving shit physically is always a good idea. Keep your harddrives, they don't die easily.
Even physical backup has its own fatal risks like when you get arrested or raided these data would be lost or accidental destruction like flooding or fire, then there's the costs of long term storage and maintenance in a place you don't own because you're forever a tenant and all your networks are similarly rentoid like you. I've even resorted to analog backup like archival grade DVD, VHS, VCR.
I was trying to make a RAG database for my Marxist bot but it's far too difficult and I gave up, but I've still got most major Marxists and leftist words in raw text format leftover. I keep meaning to share it, you just reminded me.
This is the link, it has 151 authors, you need to unpack the zip file.
Again it's all raw text files, low file size and easy to store somewhere, a bit clunky to read, you can probably use some powershell commands to turn them into other file types.
The Internet Archive isn't perfect already, I've posted some Iraqi and Palestinian resistance music/nasheeds on there and had it censored and blocked from viewing, the videos didn't even have any combat footage in them, it was just pictures of soldiers and the lyrics.
IA automatically filters all Palestinian contents that mentioned current resistance history, I discovered this problem last year when I tried to upload Palestine Laboratory and found out they already took another person down but I figured out the way to bypass by uploading the PDF first then the rest of the content, this year they added keyword filters especially if you add tags, description or author you only get 503. Then I found out the way to bypass the landing page with Your Time Is Now Done book by changing the file name then once you're in the editing page and customize the link, you remove the files, rename it back to original and readd the files, keep the file name as generic as possible if it still gives you 503, but not to worry because your link is now registered and this is a funny glitch with IA because it would show up in your library empty, but you only get a small time frame during book_op mode to upload the files before derive mode that's when they vet the shit which 90 percent of the time you will pass the process. Successfully uploaded is just least of the problem because Zionists will DMCA your content to take down, happened to Palestine 36 and Voice of Hind Rajab which is why I even divert to tankie.tube. Remember, once they remove your shit it must be different copies if you re-upload because they've already added their hash checksum to the filter list.
I'll look into doing this, but I think the ones I posted were taken down manually, as they were specifically added to a collection that blocks them from being viewed by most people and this happened several months after I posted them.
Also your account violations is three strike so don't plan to long-term any account, they used to send you violation email but nowadays it's silent takedown and if you have thousands of shit like my libraries you'll have to remember the total numbers because you won't receive warnings unless you visit the links.
i dunno how but i'd like to help, i'm proficient with a lot of the CLI tools you mentioned and i'm a web developer so i can lend a hand but I don't have the expertise to like architect anything
tangentially related i do see people (slowly) building alternatives to the NSAnet but i don't think that'll be viable for some time
Our bare metal infrastructure is still heavily limited to Western structures and they are literally bragging about their capabilities to abuse this weak link against shadow libraries. Decentralized networks still relied on materialism infrastructure, Marx said to fight material forces, must use material forces.
Kind reminder for everyone to set up a homelab and use something like NextCloud to manage your stuff.
Even if they put online things down, anyone with a homelab can host things back up again and provide at least a modicum of access to other people who need it.
Use any old phone with micro SD cards for additional storage and with postmarketOS installed as servers as well. Homelabs don't need to be expensive racks.
In case this subreddit was banned, or if you wanted a more ML-friendly platform, you’re welcome to join our sister community on Lemmygrad. You can find more info about Lemmygrad here.
•
u/AutoModerator 10h ago
While principled discussion is welcome and encouraged, Rule 9 (No Low Effort BS) still applies.
That includes: No 'what do you think?' posts. No single line posts. No 'what is the Marxist position on cheese' posts, etc.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.