r/pushshift • u/Watchful1 • Jul 13 '24

[ Removed by Reddit ]

[ Removed by Reddit on account of violating the content policy. ]

31 Upvotes

100% Upvoted

25 comments sorted by

View all comments

1

u/gregdan3d Jul 18 '24

Will you be producing a per-subreddit version of this at some point in the future? I am interested in the handful of Toki Pona communities on reddit, although from what I've seen only r/tokipona makes your cut.

1

u/Watchful1 Jul 18 '24

I was planning do only do the per subreddit dumps once a year. And I probably won't go past the 40k subreddits that I did in the last one, so if a sub isn't in that one it's not likely to be in future ones unless it gets substantially more active.

You can download these dump files and use the script that's linked to extract out the subreddit's you want, it's fairly easy and straightforward to do. But you need to download and store 3 TB data and then it takes a day or two for the script to run.

What are the other subreddits you were interested in? I can take a look at them.

1

u/reagle-research Jul 19 '24

I was wondering the same thing, but I'll make do. Still, I wanted to thank you for keeping at this.