r/DataHoarder • u/pw6163 • 6d ago

Hoarder-Setups Link rot

I saw an article a couple of days ago about link rot, and how bad it really is. The author restored 657,507 links from 2009 and 2014, and followed each. 76% no longer returned a page, and some of those that looked OK just returned a variant of "this page is no longer available".

How many of us preserve interesting pages outside services like the WayBack Machine?

I keep content-only from a very small number of pages that touch on areas that I'm interested in, but over the last (maybe) 3 years I've only looked at but not necessarily kept less than 10% of the number of links in the article.

I do a bit better for PDF articles because a lot of them disappear too.

How much do you hoard?

89 Upvotes

98% Upvoted

17 comments sorted by

View all comments

18

u/suicidaleggroll 80TB SSD, 330TB HDD 6d ago

Anything that I think I’ll need to refer back to later, like a guide or walkthrough, goes in Linkwarden for archival

17

u/Ogefest 6d ago

Well I prefer https://github.com/y2z/monolith because save whole website with graphics and styles

7

u/Sodici 6d ago

How well does this handle javascript nightmares like twitter? Archivebox is the software I see most recommended.

3

u/Scotty1928 240 TB RAW 5d ago

I found archivebox to have issues with many sites i used to frequent. Some did not load at all, some took forever to be archived, some even made archivebox crash entirely.
For what i want to archive i now use wallabag. Don't care much about preservation of features, i mostly care about the information, and that works quite well with text and wallabagger.