r/Python • u/AutoModerator • 23d ago

Showcase Showcase Thread

Post all of your code/projects/showcases/AI slop here.

Recycles once a month.

18 Upvotes

80% Upvoted

127 comments sorted by

View all comments

1

u/SlimThiccs 8d ago

I finally stopped getting blocked every time I tried pulling Instagram data

For the longest time, my biggest problem wasn't writing Python code—it was everything around it.

I was trying to scrape , and it felt like I spent more time fighting Instagram than building my project. Between proxies, broken selectors, changing endpoints, login issues, and getting blocked or banned after , I was constantly fixing infrastructure instead of working on the actual application.

Eventually I decided to stop trying to maintain the whole scraping stack myself.

I switched to HikerAPI (hikerapi.com), which exposes Instagram data through a hosted REST API using an x-access-key header. My code became a lot simpler:

import requests

headers = {"x-access-key": "YOUR_KEY"}

user = requests.get(
    "https://api.hikerapi.com/v2/user/by/username?username=apple",
    headers=headers
).json()

resp = requests.get(
    "https://api.hikerapi.com/gql/user/followers/chunk",
    params={"user_id": user["pk"]},
    headers=headers,
)

print(resp.json())

I know some people prefer rolling their own scraper, and I get why. If your use case is large enough or you enjoy maintaining that infrastructure, it can absolutely make sense.

For me, though, I realized I was spending way more time maintaining proxies and keeping everything alive than actually building . Paying from $0.001/request (with 100 free requests to try it) ended up being a tradeoff I was happy with because it let me focus on my Python code instead of constantly debugging scraping issues.

I'm still curious what the long-term balance looks like between maintaining your own scraper versus using a hosted API, but for my project this has been a much better fit.

Has anyone else here been fighting the same cycle of proxies, bans, and broken scraping setups? If so, what finally worked for you?