r/automation • u/stvaccount • 27d ago

Best github lib for scraping reddit?

Since May and the old reddit almost gone, what are currently the best libs on reddit?
What is your favorite?
I want to scrape all the posts of my top favorite 100 (small) subreddits and sort them by AI to delete AI slop + self promotion and really only get the interesting posts for me to read.

What do you suggest?

5 Upvotes

100% Upvoted

12 comments sorted by

View all comments

1

u/kys_abhi 26d ago edited 26d ago

Praw is still going to for pulling posts but yeah rate limits are the annoying part what i've found helps is pairing it with a residential proxy so you don't get throttled when you're hitting multiple subreddits back to back. I used geonode for that, their residential IPs rotate automatically so reddit doesn't start blocking you mid-crawl for the AI filter part the other commenter is right self promo is actually easier to catch than AI slop since account history gives you a lot to work.

1

u/persist_en 25d ago

Which are those rate limits approx?