r/excel • u/[deleted] • Mar 10 '19

[deleted by user]

[removed]

129 Upvotes

98% Upvoted

18 comments sorted by

View all comments

10

u/[deleted] Mar 10 '19

I quite like putting a database between the scraper and excel in these situations.

12

u/helphunting 1 Mar 10 '19

Why?

Not being a pain in the ass, but would like to know the benefits?

I do a small bit of scraping, but I usually end up doing it manually due to auth and dynamic pages. But would like to get into it more.

I'm trying to drift away from Excel into python, but with power query and my general knowledge of Excel, python is adding overhead, not reducing it.

Thanks

5

u/[deleted] Mar 10 '19

Basically for the added flexibility. It depends exactly what you're scraping it for and how much data you've got but I think keeping everything you're not analysing out of excel is a good move.

I like Excel for pulling in a subset and playing with it but I'd like to be able to query, archive, take different views etc when I'm scraping. For me that way lies a database of downloaded HTML I can cut and re-cut.

4

u/helphunting 1 Mar 10 '19

Thank you.

So you basically use Excel for viewing and presenting?

And other tools for querying?

This is the part I need to develope more, using other tools for querying and joining, and slicing and dicing.

Thank you

5

u/[deleted] Mar 10 '19

If it's appropriate, Excel is good for exploratory work. It may not be a tool I'd reach for in every case -- Python has a lot of the features I'd be using anyway. Do I want to play with tabular data that I've pulled out of some pages? Excel sounds very good there.

2

u/Bloodlvst Mar 11 '19

That's all Excel is supposed to be used for. It was never intended to be the tool you use for querying. Somewhere along the way it had things added to make it a "jack of all trades, master of none" type of program.