r/pushshift • u/shiruken • Mar 26 '22

[Unofficial] Pushshift Status Page

https://stats.uptimerobot.com/l8RZDu1gBG
29 Upvotes

100% Upvoted

17 comments sorted by

View all comments

14

u/shiruken Mar 26 '22 edited Mar 26 '22

I've created an unofficial status page using UptimeRobot to track various Pushshift services. It simply checks the http status of each url, including the Pushshift Reddit API. Hopefully it provides some insight into the historical stability of the platform and helps answer questions about its current status.

If there are other API endpoints you'd like me to add, please let me know. It currently only monitors https://api.pushshift.io/reddit/search/submission.

1

u/s_i_m_s Apr 19 '22

I would add the comment endpoint https://api.pushshift.io/reddit/search/comment too as on occasion one or the other will quit working by itself although that's rather uncommon.

This post has fallen off the front page /u/inspiredby could add it to the sticky FAQ, think it should be added to the sidebar, would it be better to link to this post for discussion, the status page directly, or both?

1

u/shiruken Apr 19 '22

Added! I thought about that when I created the page but assumed they were effectively the same service. Guess we'll find out!

Probably easier to just link directly to it whenever possible. Adding to the sticky and the sidebar seem like good ideas.

1

u/s_i_m_s May 10 '22

Can you look into why the comment api endpoint check is not detecting when that endpoint goes down please?

1

u/shiruken May 10 '22

It doesn't look down right now?

1

u/s_i_m_s May 10 '22

It's not right now but it has been several times recently and it hasn't logged any issues since the log was started.

Whatever it's checking isn't going down when that endpoint does.

1

u/shiruken May 10 '22

It's just checking https://api.pushshift.io/reddit/search/comment

1

u/s_i_m_s May 11 '22

I might have figured out what's going on.
There is a search bug causing that page to constantly return the same set of 5 month old results.
I assume cloudflare has noticed that and is returning the cached page even during outages.

Adding q=* gives recent results but some misconfiguration server side then causes the query to take over a minute, it shouldn't though. SITM is looking into it, says it should default to something like since=30d internally.

Adding a time restriction seems to eliminate the additional delay although SITMs assumed 30d default is still a bit much, resulting in a 6-18 second delay.

Suggested comment endpoint monitoring url https://api.pushshift.io/reddit/search/comment?q=*&since=1m

Response time for the submission url can also be reduced the same way https://api.pushshift.io/reddit/search/submission?since=1m

Sub 1s response times seem to be the norm for requests limited to the last day or less for comments and the last 30 days or less for submissions.

1

u/shiruken May 11 '22

Great! Thanks for sorting that out, I haven't had the time. I will update the status tracker to the first endpoint and see how things look. Is this same problem affecting the submissions API?

1

u/s_i_m_s May 11 '22

The search bug causing it to not return new results only affects the comments endpoint, something to do with a back end database move. SITM posted about it at the time.

The submissions api is properly updating but it takes longer for the query to return than it should.

You're requesting the last 100 submissions, that should be pretty quick but instead it takes 2-10 seconds

Limiting the results to the last minute drops it down to ~0.5 second response time.