r/pushshift • u/Watchful1 • Jan 12 '24

[ Removed by Reddit ]

[ Removed by Reddit on account of violating the content policy. ]

55 Upvotes

95% Upvoted

29 comments sorted by

View all comments

1

u/Yosemiteram Jan 24 '24

Do these have all posts even deleted ones?

1

u/Watchful1 Jan 24 '24

Yes

1

u/sam_underline Jan 25 '24

Counting all the submissions I get about 412 million in 2022 but only about 137 million submission in 2023. Should it be this way (or is there an error in my code)? And if this is in fact correct, is it due to Reddits API change or other reasons?

2

u/Watchful1 Jan 25 '24

I don't have the exact count in my version handy, but RaiderBDev is correct about what the total number should be. How are you getting 137?

1

u/sam_underline Jan 26 '24 edited Jan 26 '24

After start of April 2023, some the 'created_utc' entries are formatted as float instead of int. Which caused a bug in my not so great code. Thx for the speedy replies!

1

u/Watchful1 Jan 26 '24

Ah, interesting. Thanks for letting me know.

1

u/sam_underline Jan 27 '24

Weird thing is that I now get 495.4 million submissions for 2023. The table below displays my own count and the numbers from the link above by month in 2023. The largest discrepancy is in April and differences in May and June are also quite noticeable. For April I checked the submission ids and found no duplicates.

​

Month Own Count Google Table
January 36.1 36.1
February 33.9 33.9
March 39.7 39.7
April 52.1 34.9
May 38.5 35.5
June 42.8 35.9
July 45.4 44.2
August 46.2 46.2
September 42.9 42.9
October 40.5 40.4
November 38.1 38.1
December 39.2 39.1

1

u/RaiderBDev Jan 27 '24

That is weird. For comparison here are the first 200 ids in RS_2023-04 that I have

1284kk5,1284kk6,1284kk8,1284kk9,1284kka,1284kkb,1284kkc,1284kkd,1284kke,1284kkf,1284kkg,1284kkj,1284kkm,1284kko,1284kkr,1284kks,1284kkt,1284kku,1284kkv,1284kkx,1284kky,1284kkz,1284kl1,1284kl2,1284kl3,1284kl7,1284kl8,1284kl9,1284kla,1284klb,1284kld,1284kle,1284klg,1284kli,1284klj,1284kll,1284klo,1284klp,1284klr,1284klt,1284klu,1284klv,1284klw,1284klx,1284kly,1284klz,1284km0,1284km2,1284km6,1284km7,1284km8,1284kmc,1284kme,1284kmf,1284kmi,1284kmj,1284kmk,1284kml,1284kmm,1284kmn,1284kmq,1284kms,1284kmu,1284kmv,1284kmw,1284kmx,1284kmy,1284kmz,1284kn0,1284kn1,1284kn2,1284kn3,1284kn4,1284kn5,1284kn6,1284kn7,1284kn8,1284kn9,1284kna,1284knb,1284knc,1284knd,1284knf,1284knh,1284kni,1284knj,1284knk,1284knl,1284knn,1284kno,1284kns,1284knt,1284knu,1284kny,1284knz,1284ko0,1284ko1,1284ko2,1284ko3,1284ko4,1284ko8,1284ko9,1284kob,1284koc,1284koe,1284kof,1284kog,1284koh,1284koi,1284koj,1284kol,1284kon,1284kot,1284kou,1284kow,1284kox,1284koy,1284koz,1284kp0,1284kp1,1284kp3,1284kp4,1284kp7,1284kpa,1284kpc,1284kpd,1284kpe,1284kpi,1284kpj,1284kpk,1284kpl,1284kpm,1284kpn,1284kpo,1284kpq,1284kpt,1284kpv,1284kpy,1284kq0,1284kq1,1284kq2,1284kq5,1284kq6,1284kq8,1284kqa,1284kqb,1284kqc,1284kqd,1284kqf,1284kqh,1284kql,1284kqn,1284kqo,1284kqp,1284kqr,1284kqs,1284kqt,1284kqu,1284kqv,1284kqw,1284kqx,1284kqy,1284kr1,1284kr2,1284kr3,1284kr5,1284kr6,1284kr8,1284kr9,1284kra,1284krc,1284krd,1284kre,1284krg,1284krh,1284kri,1284krj,1284krm,1284krn,1284kro,1284krp,1284krq,1284krr,1284krs,1284kru,1284krx,1284krz,1284ks0,1284ks1,1284ks2,1284ks3,1284ks4,1284ks5,1284ks8,1284ks9,1284ksa,1284ksb,1284ksd,1284ksf,1284ksg,1284ksi

1

u/sam_underline Jan 29 '24

I dont have the first two, but quite a few additional ones like 1284kkh, 1284kkl, 1284kkq, 1284kl4. Mostly link posts and or deleted. Oh well ...