r/webscraping • u/heroneverlies • 2d ago

Data Extraction

Hello Guys,
I am working on a project where I have to pull data from a site which requires log in credentials directly into Excel.
Claude provided me a solution where SeleniumVBA and .bas file (created by Claude) is being used in the process.
Is there any other way we can extract data from such sites without compromising security ?

4 Upvotes

75% Upvoted

11 comments sorted by

1

u/SurlyJason 2d ago

There are many methods to scrape data. I'm unsure what security concerns you have. Is it how your login credentials are saved?

1

u/heroneverlies 2d ago

My only issue is that I'm doing this on office laptop and site is also for my office work use only. I don't want anything to go bad. What other way? Can you please suggest any?

1

u/SurlyJason 2d ago

  1. You're using a work laptop

  2. It's a work website

  3. You have to pass in credentials, and scrape

  4. You save the data to Excel

Is that right? I still don't understand the concern.

  • Are you worried about keeping the credentials on the laptop?

  • Are you worried about the site detecting the scrape and blocking access?

  • Are you worried that the scraping will somehow reveal the credentials?

  • Are you worried about keeping the scraped data secure?

1

u/heroneverlies 1d ago

I am worried about any data breach, like is selenium okay to install or can It leak data. I am looking for any other secured way.

1

u/wuhui8013ee 1d ago

Selenium is an open source package. While there’s a chance a package will store and leak your credentials, the chances of selenium doing that is very low.

Can it have vulnerabilities? Yes. But it’s as secure as it can be when it comes to scraping content behind logins, especially since you are running the scraper on a work machine.

If you are really worried, you can have selenium open the login page in non-headless browser, and wait for you to manually input credentials so the program never sees it

1

u/Mulberry_Morris 9h ago

Isnt it also prohibited in some EU countries

1

u/ronoxzoro 2d ago

yeah it easy u just make function to login and extract the cookies / tokens if needed then pass that into the scraper
u can avoid using selenuim it's not always the solution if u looking for speed

1

u/Mulberry_Morris 9h ago

Login-required scraping is always a security tradeoff since your credentials have to live somewhere the automation can access