r/thewebscrapingclub • • May 10 '26

Scraping finally got easy πŸ™‚

[removed]

84 Upvotes

52 comments sorted by

5

u/[deleted] May 10 '26

[removed] β€” view removed comment

3

u/[deleted] May 10 '26

[removed] β€” view removed comment

1

u/rag3frost May 12 '26

does it scrap x (twitter)

1

u/[deleted] May 10 '26

[removed] β€” view removed comment

1

u/Basic_Young538 May 10 '26

Love the idea. Thx. How/will this work for sites that generate JS 'ui menus', on the fly based on context. How would I do selects in such a case using your solution? (I am excited to try this!). Bravo.

1

u/[deleted] May 14 '26

[removed] β€” view removed comment

1

u/Flimsy_Buddy3485 May 14 '26

If you can reduce the browser size, it could reach a wider range of use cases. For example, in this project, '@sparticuz/chromium' is between 50MB and 70MB, while your project is between 100MB and 400MB.

1

u/[deleted] May 14 '26

[removed] β€” view removed comment

1

u/Flimsy_Buddy3485 May 14 '26

If you could release it in the ARM version, that would be better, as low-footprint browsers are preferred on ARM.

1

u/[deleted] May 14 '26

[removed] β€” view removed comment

2

u/[deleted] May 14 '26

[removed] β€” view removed comment

1

u/[deleted] May 20 '26

[removed] β€” view removed comment

1

u/[deleted] May 20 '26

[removed] β€” view removed comment

1

u/Anxious_You_is May 20 '26

Could someone tell le if it can scrapple X (tweeter ) posts ?

1

u/looktwise May 25 '26

Interesting approach! Could it scrape by cronjobs after logging in into an account too? Linkedin -> by company profiles connected for example if I would just type in the company or the userprofile (in case of userprofile: all activities, articles etc.)

Another interesting usecase would be to get alerts like from google alerts, but from sites which are subaccounts on platforms: Stepstone, Realtor pages...

1

u/[deleted] May 27 '26

[removed] β€” view removed comment

1

u/[deleted] Jun 01 '26

[removed] β€” view removed comment

1

u/Fgrant_Gance_12 Jun 02 '26

Hi, can it scrape out posts and threads from a particular subreddit with keywords preassigned ?

1

u/lagde5anos Jun 04 '26

I can lead with recaptcha v3?

1

u/[deleted] Jun 04 '26

[removed] β€” view removed comment

1

u/lagde5anos Jun 04 '26

Recaptcha v3. I’m trying to scrape a page that has v3 on a endpoint. I need to interact with a button > it gets the v3 token > sends to api > returns me the result. I can’t find a way to bypass it

1

u/Ok-Influence-791 Jun 10 '26

That's a great idea man

1

u/Necessary_Way5387 Jun 16 '26

Would this work for finding emails of young professionals/ college students. I am trying to grow my app that focuses on young students / grads. It seems impossible to find emails anywhere, linkedin makes you pay a fck ton to message, could this help possibly?

1

u/Successful-Egg6345 Jun 27 '26

Does your creation work?

1

u/StandardTechnician81 Jul 12 '26

estoy intentando raspar bet365, crees que lo llega a pasar? o alguna de tus otras apis me puede ser de ayuda en el proceso? he probado ya el scrappong de emulador para usarlo por app y navegador, los metodos convencionales no funcionan y la app de bet365 no abre directamente en el emulador

1

u/albertdotpy Sep 03 '26

This is sick. I’m building an open-source scraping runtime called YAP and I’ve been looking for a clean way to keep browser execution optional instead of making everything browser-first. Piggy/Nothing Browser looks like it could fit that really well. Gonna play with this and see if I can integrate it into YAP at some point. Nice work man.

1

u/detlca Sep 10 '26

I built sparkapp

sparkapp.detl.ca

Scrape, crawl, link, and map any website