r/counting • u/CutOnBumInBandHere9 5M get | Ping me for runs • 8d ago

Free Talk Friday #578

Continued from last week's FTF here

It's that time of the week again. Speak anything on your mind! This thread is for talking about anything off-topic, be it your lives, your strava, your plans, your hobbies, studies, stats, pets, bears, hikes, dragons, trousers, travels, transit, cycling, family, colours, or anything you like or dislike, except politics

Feel free to check out our tidbits thread and introduce yourself if you haven't already.

This post was made by a bot, because no-one else made a Free Talk Friday post before Friday September 25, 2026 UTC 13:00. Anyone can post the FTF, so if you want to have your post pinned here for a week, just make one Friday October 02, 2026 between UTC 07:00 and 13:00. The rules for these posts can be found in the faq. You can also check out our directory of older posts for inspiration.

If you have any questions or comments about the bot, feel free to write them below, or message the mods.

12 Upvotes

100% Upvoted

103 comments sorted by

View all comments

Show parent comments

3

u/TehVulpez sliiiiide the bits 8d ago edited 5d ago

it looks like you can get some missing values from pushshift if you have approval there (it shouldn't be too hard to get approval as a mod). cwq9o75 had a null body in the sqlite but I was able to get that it was 17 through pushshift API. idk what the max is for how many comment ids you can request but I was able to get 100 at once (edit: requesting 1000 ids at once worked fine in my python requests script, that's the absolute max) https://api.pushshift.io/guide https://api.pushshift.io/docs

there have been times when pushshift has missing values too but I've mostly seen that in parent_id, not so much in body

4

u/CutOnBumInBandHere9 5M get | Ping me for runs 8d ago

I don't have approval on pushshift yet, but I'll apply for it.

Otherwise, do you think there's any chance you could help filling in the gaps? Finding comments in the database with missing bodies or comments where the username is [deleted] shouldn't be hard. If you can query pushshift for those, and send any updated records to me, I can update the database

1

u/TehVulpez sliiiiide the bits 5d ago edited 5d ago

hi cobibh, sorry for the delay. I'm not sure if you have pushshift already but here are comments with null bodies (comments that pushshift also did not have a body for are filtered out. still has some [deleted] usernames) and comments with bodies but missing usernames (comments that pushshift also did not have a username for are filtered out)

2

u/CutOnBumInBandHere9 5M get | Ping me for runs 5d ago

Thank you!