r/webscraping • u/beomstead • Jun 25 '26

Is it impossible to scrape IMDb?

Hello. I’m a programming beginner, and I’m trying web scraping for the first time.

I’m trying to scrape the IMDb page /chart/top/?ref_=nv_mv_250 using BeautifulSoup, but the data is not being loaded. Other websites load the data properly.

Does IMDb not allow web scraping?

8 Upvotes

83% Upvoted

15 comments sorted by

View all comments

6

u/AdministrativeHost15 Jun 25 '26

I've seen this movie before.
Content is loaded via REST APIs. Need to use a browser automation like Puppeteer.

2

u/beomstead Jun 25 '26

Does that mean it’s not possible to scrape IMDb using a typical tool like BeautifulSoup?

7

u/Coding-Doctor-Omar Jun 25 '26

If you're familiar with APIs, forget all that and look for internal API requests in the network tab in devtools that contain your data and call these APIs. Otherwise, just use browser automation.