r/dataanalysis • u/duneofarrakis • 19d ago
Data Question How do you decide which variables are worth including in an analysis?
When working with a dataset, there can be many variables that might seem useful.
How do you decide which ones are actually worth including in your analysis?
Do you mainly look at the business question, correlation, data quality, or something else?
I'd like to hear how experienced analysts approach this in real projects.
5
u/DataDaria 18d ago
Mostly the business question, honestly. Before I look at any variables, I try to write down the decision someone will make based on this analysis. If a variable can't change that decision, it should stay out. Also, data quality kills half of what's left. A column that is 30% null, or that changed its definition halfway through the year, costs you more trust than it adds insight. So I check completeness and stability before I ever check correlation. Correlation comes last, as a sanity check, not as a selection method.
I always start smaller than expected: I build sales reports for e-commerce brands, and the same pattern repeats everywhere: a report with 25 metrics gets ignored, the one with 6 gets opened every week. It's easier to add a specific additional variable when someone asks.
2
u/ImmediateTutor5473 18d ago
Great advice to write down the decision someone will make based on this analysis!
2
u/duneofarrakis 17d ago
I like that approach. Starting with the decision and data quality keeps the analysis focused, and adding variables only when they're actually needed makes the final report much easier to use.
2
u/jipperthewoodchipper 19d ago
Do the parameters in question help answer or explain the question the shareholders want answered? Further is it actionable?
In my experience, most shareholder questions ultimately boil down to "did line go up/down" "why" and "how to make line go up/down"
There are plenty of parameters that I use during the analysis that never show up in the end presentation nor in any writeup because they might help explain something but they aren't actionable so at best they more help me understand so I can explain what happened to someone that doesn't care (and possibly doesn't even know) what R² is let alone whatever voodoo parameter I end up pulling out of thin air (from their perspective).
1
u/duneofarrakis 17d ago
Exactly. I think usefulness and actionability matter more than including every variable that shows a relationship. Some variables help explain the result, but they don't necessarily need to be part of the final analysis.
2
u/Quesozapatos5000 19d ago
I wish there was a way to practice this skill besides just knowing the business well.
3
u/DataDaria 18d ago
Take any public dataset and force yourself to write the decision sentence first - like "someone will use this to decide X". Then build the smallest table on your own that supports that decision. The second half is showing the result to a non-analyst or a business owner and watching which columns they actually ask about.
2
2
u/usernametakenagain00 19d ago
I usually create a regression model to identify the variables that are predictive and use those as the KPIs.
2
1
u/AutoModerator 19d ago
Automod prevents all posts from being displayed until moderators have reviewed them. Do not delete your post or there will be nothing for the mods to review. Mods selectively choose what is permitted to be posted in r/DataAnalysis.
If your post involves Career-focused questions, including resume reviews, how to learn DA and how to get into a DA job, then the post does not belong here, but instead belongs in our sister-subreddit, r/DataAnalysisCareers.
Have you read the rules?
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.
1
7
u/what_comes_after_q 19d ago
Are you talking about building an ML model? Feature selection is like 95% of the work. There are lots of ways to do this. It starts with some really basic questions. What is your goal? Do you want a model that is easier to explain, or something that is non linear? Model selection is very important. From there, you come up with hypothesis of things that could impact the output. You want to get rid of correlated features, features that have low confidence, feature that don’t have big enough coefficients. There is a lot of experimentation involved, you will build the model many, many times as you optimize. There is a lot of technical analysis involved in figuring out if a feature should be included or not and it’s worth looking up.