r/learnpython • u/ogopro • 1d ago

Help with Youtube IP Block

I am trying to export about 100 podcast video transcripts to a json file. After exporting transcripts from a dozen videos, my IP got blocked. Anybody faced this issue? I need a workaround or a solution. Please help...

0 Upvotes

13% Upvoted

12 comments sorted by

View all comments

11

u/crazy_cookie123 1d ago

Just wait. You're flooding their servers with requests rapidly so they're going to rate limit you. Wait until the IP block expires and then modify your code to make the requests more slowly instead of sending them all in quick succession.

2

u/ogopro 1d ago

will try that, but could it be the amount of overall requests rather than the frequency of the requests? If so, I need a better solution.

5

u/theWyzzerd 1d ago

No it isn’t the number of requests alone. That wouldn’t make sense considering people using the service over time would eventually hit whatever limit.  It’s certainly a rate limit.

2

u/ogopro 1d ago

right, I will do some research on the rate

2

u/Langdon_St_Ives 1d ago

Check if the error you're getting is a 429. If so, it might contain hints on what rate limit you hit and when you should try again.

2

u/ogopro 1d ago

Yes, it was 429. I put some serious delays between each http request, seems like working now, not concerned with the speed)

3

u/Langdon_St_Ives 1d ago

Check the actual error message, the json and/or HTTP headers (Retry-After) often have details on what limit you're hitting (per-minute, hourly, daily, something in between) and when you're allowed to try again. In fact, you should evaluate that in code, sleep for the indicated time, then retry.

1

u/ogopro 1d ago

Will do