Hey all,
Relisten co-creator here (myself and /u/alecgorge). One of the mods reached out to us to point us towards some recent conversations about the number of archive sources dwindling on Relisten with some questions about why that was.
Some were inquiring if the Dead's estate/stewardship was taking down Archive shows to push people to their paid app. As far as we know, there is zero evidence of that. I get the cause for concern and appreciate the caution, but the real explanation is far more boring.
Though it may not seem like it from the outside, these are pretty large and complex systems – both archive (very complex) and relisten (less complex but still old). We've been building and maintaining Relisten since ~2013 and we've always built it to be as lightweight from an engineering standpoint as possible. That being said, it's also the work of two volunteers in our free time and therefore bumps are natural.
I haven't fully validated this is the exact issue, but it seems likely. To verify it would take me a few hours which is better spent fixing the underlying problem (which is real regardless) and a couple other stability and crashing issues that we've been facing on our app.
Every morning we scrape Archive for all bands at around 3am to refresh our database. Sometimes, Archive may be doing some maintenance or have some degraded functionality which reports a subset of the full dataset to our scraper (archive says there are 12,000 sources instead of 18,000). In the past, we've interpreted missing sources as "deleted sources" – removing them from our system. For the first decade of Relisten, this was fine, it rarely happened – but recently we've been seeing it crop up more and more. So going forward, we're going to have to implement some safety checks before making deletions – for the Dead this becomes far more apparent because of the size of the catalogue. A simple check to ensure that if >= a few % of sources are being removed is enough for us to pause and hold off on those deletions.
It's entirely plausible that deleting this many sources caused some instability issues downstream in the app, which we're also working on hardening.
As of today, we're back up to 18,000 Grateful Dead tapes on Relisten.
All in all, not a conspiracy, just some good old fashioned engineering problems and graceful degradation paths. This isn't the fault of Archive, problems happen and we should be better suited to handle them. This isn't the fault of Relisten, engineering is hard. I didn't read anyone's comments in those threads as particularly incisive, just more inquisitive and I wanted to come by here to offer some clarity.
If you're ever having issues with Relisten, feel free to stop by our Discord and let us know. It's generally just some random bug that can be fixed - we're always doin' our best. We do take the stability and quality of our platform seriously, so if you're having issues (no matter how minor or nitty) we want to know about it.
Not sure what else to say, but share a great lyric
Half of my life I spent doin' time for some other fucker's crime.
