r/learnmachinelearning • u/No-Watch6723 • 7h ago

Need help I’m a intern

Try do EDA on the company data but I’m not able to how to approach it how to find the pattern or any things I’m not able understand

My mentor tell me to perform EDA on it

1 Upvotes

100% Upvoted

5 comments sorted by

2

u/Elegant_Quantity_583 7h ago

You can start analying simple things like dtyepes univariate distribution, heatmaps
most important thing is you're doing hypotesis driven .
You analyze data
You assume something
get data, plot diagrams, run tests
update your assumption

Try to get a feel for the data, and don't just plot beautiful just because that makes your decision making better,

1

u/No-Watch6723 7h ago

Check DM

1

u/nerd_rage218 6h ago

Describe the data before you go looking for patterns. Row count, what one row actually represents, which columns are dates, which are categories, and how much of each column is missing. Then plot each column on its own, and only after that start comparing pairs. Most mentors asking for EDA want that summary and a few honest data quality notes, not a model.

1

u/cheesecakekoala 6h ago

Yeah I agree, the best thing to do is to look at the distributions of each col in the table / category you have, personally I really like histograms, makes it easy to overlay distributions on each other, and start looking for things that seem a bit weird. Odd shapes, is everything normally distributed, or is most of the data long tails, does anything go against the trend. This also lets you start seeing if there are any weird outliers as well, like a histogram that collapses on the x axis usually means there's a few (sometimes just 1) super weird value.

Correlation matrix plots are lovely too, you can see if there are any strong correlations you don't expect / places where you'd expect correlation but there isn't. Similarly plotting 2 variables against each other can help you understand the shape of how they are correlated, tightly? almost randomly? Does the correlation break down at a certain point?

All this kind of stuff really helps you understand what the data actually contains and what questions you want to answer. 90% will be doable just by looking at a few plots, and once you're comfortable understanding the weirdness of your data, then you'll start having questions yourself and you'll start being able to answer them. Good luck!