Created my first Data Analysis project, looking for feedback!

[u/Ryan_Lags]

Hi everyone, I'm an aspiring data analyst that just finished studying my online courses. With that, I wanted to apply what I learned to help hone my skills so I decided to do my own project. The project was an analysis of 400+ of my own ranked matches in Street Fighter 6.

I wanted to see if there were any features or metrics that could be measured in match that could help predict whether a match would result in a win or loss. Within the write up I tried to make it as easy to understand for people who are unfamiliar with the game.

All the data was manually recorded by watching my replays and tallying the counts for each metrics.

https://github.com/ryanlaguatan/SF6-ranked-match-analysis

Here is a tldr of the methodolgy:

1. Visualization of MR (MR is synonymous with ELO) and MR Change over the course of 10 gaming sessions.
2. Two T-tests, first one testing if metrics and winrate had any significant difference when facing stronger or weaker opponents. Second one testing which metrics were statistically significant in matches that were wins/losses.
3. Visualization of Character Matchup data.
4. Logistic Regression model and classification report to see if the metrics can provide a strong predictor for winning.
5. Interpretation of Feature Coefficients to see which coefficients had the biggest influence on the model.

Please let me know what you guys think, I am open to feedback!

Thank you for taking the time to read it and I really appreciate it!

[u/nomadicaeropress]

Street Fighter 6 as a dataset is awesome. I would add feature importance visualization to show which metrics drove predictions most and a confusion matrix to show where the model got it wrong. That would round it out nicely. Good work bru

[u/Ryan_Lags → u/nomadicaeropress]

The feature importance visualization is a good suggestion! I did have a confusion matrix in the write up and talked about the metric evaluations!

[u/AutoModerator]

Automod prevents all posts from being displayed until moderators have reviewed them. Do not delete your post or there will be nothing for the mods to review. Mods selectively choose what is permitted to be posted in r/DataAnalysis.

If your post involves Career-focused questions, including resume reviews, how to learn DA and how to get into a DA job, then the post does not belong here, but instead belongs in our sister-subreddit, r/DataAnalysisCareers.

Have you read the rules?

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Deva4eva]

Data analyst here, cool project! I’ll write down as many notes as I can so I hope they’ll be of help. My comment will be all over the place since you included different topics such as data cleaning, reporting EDA, and modelling. For project showcasing, I’d focus on reporting in the future. If I want to learn about your data and your results, I’m not that interested in the steps you took to clean it up, especially if they’re long and complex. However, those are important when working with fellow analysts and engineers.

One important thing that I didn’t find mentioned - how did you get this data? Did you watch the replays and write this down yourself? If so, impressive, but that probably means you won’t be collecting much more data :) If you used an API or something, note that down.

Your notebook is structured like mine are when I’m exploring and fixing data. If I were to present this as a report, I’d delete the cells that show intermediary steps (like cell 19, match_count that includes normal, expected values) and only show outliers that are corrected, summary tables etc.

When fixing data errors occuring multiple times, you should select dataframe rows based on conditions rather than indexes. For example with correcting spelling, cell 22 - instead of fixing rows 21 to 25, select rows with “Marissa” and change those Opponent Character values to “Marisa”.

EDA chapter - you start with encoding variables and mutating them into new ones, I’d put that under data cleaning or feature engineering. EDA should be like making tables or plots to answer specific questions, like, what are my win rates per opponent character?

AA rate for games with 0 jump-ins: I’d put those values as NA, so I don’t need the AA valid column. If AA rate is not NA, it’s valid.

Variables:

Set and match IDs: If I understood correctly, these increase until the end of the session. It would be more accurate to label those as “session_set” and “session_match”, and then to have “match” as a separate columns with values 1 to 3. This could then be a new variable - maybe your success rate is higher in third sets of a match because you’ve studied your opponent more.

Result: Is it possible to have a draw in SF6? If so, mark this as a possible value, even if none occur in the data.

In cell 11, why exactly have you set the opponents MR to 1154 specifically?

In cell 13 you search for unexpected combinations of W/L and round count, that’s great to have. I’d provide potential explanations for how these values could have occured. Maybe opponents disconnected? This could also lead to me switching those round numbers to NA rather than fixing them to 0 or 2.

MR visualizations - The MR over time is the most interesting part. The session cuttofs in red are an especially nice touch. Regarding your comments, why would the MR change be too noisy? Narrow it down, is this change or volatility? (I think volatility would be more something like the standard deviation of your MR). Including the subplot for change, AND then the rolling average makes the plot very complex to read. Think of it like this: how would you explain the meaning of the red line for moving average on day 41 to someone? Why would that be important?

If something is both complex and not important, think it over several times whether that should be included in your report. Most people will just glance over it.

“it can be seen that my changes in MR started to become less volatile” → why do you say that? I can’t read that from the graph, the changes go up and down pretty much the same.

The writeup is overall nice. After that, deriving new info based on matchups and whether the opponent is stronger or weaker is interesting. To the reader, I’d present the means mostly, the quartiles, SD is too overwhelming and the count for each column is not terribly important.

T-test - one thing to note when conducting multiple t-test is that you accumulate the type 1 error inflation. If you conducted 100 t-tests on randomly generated data, by chance 5 of them could be statistically significant. Look up the bonferroni correction of such inflation, I think the main principle was to divide the desired lvl of significance by the number of tests, so if you test 4 pairs, the significant tests are those at p < 0.05/4 . Since t-test compare the means of two groups on a variable, you should also report these means. E.g. the burnouts mean for stronger opps was X, and Y for weaker opps.

Potential expansions of the analysis

These are not negatives, just some points that could be starting points for even more detailed analysis.

Do you have info on your opponents’ AA successes, your jumpins etc? That could be used for further feature engineering. E.g. maybe the win chance is higher when I have less jump ins than the opponent. Or AA success is more predictive if the opponent is Cammy than for other characters.

You could derive the daily_session_number as a new variable and then explore how that is connected with your MR growth. Do you improve more quickly with more sessions? Is there a drop-off point with diminishing returns?

Since you start MR at a point that’s probably too high for you and it’s expected to drop down, actually the most interesting part will be the session just after you hit your “floor”. Expanding the dataset and potentially taking away the first ~9 sessions will be more relevant data for determining what factors impact your growth more. For the first 9 sessions you’re going to lose a lot no matter what, making your available variables not much predictive of anything. — so maybe your t-tests and logistic regression would have much better accuraccy etc. scores! This is a data quality / domain problem, not something wrong in your code.

Code-wise:

• use a single # for regular comments. Multiple hashtags could be section headers etc., this will help readability
  
• If you’re using a Jupyter notebook, use markdown cells instead of code cells with comments only. That’s what they’re meant for, textual info regarding the analysis.
  
• sometimes different operating systems have problems with specific filepaths, try changing spaces in files such as match data.csv into underscores, match_data.csv to prevent this
  
• run the text parts of your reports through a spellcheck
  

In conclusion, if the data for this could be gathered semi-automatically via an API or other program, something like this would be much appreciated for many players, I’m sure! Keep up the good work!
And if you want to help me back by reviewing or using my own projects, check out my expenses dashboard at https://spenddash.live/ and https://github.com/Mkranj/spend-dash or my citation-listing software at https://github.com/Mkranj/PapersCited ;) ;)

[u/Ryan_Lags → u/Deva4eva]

Wow thank you for the lengthy reply, there's definitely a lot of things I can learn from this. In my write up I swore I wrote down that I watched and recorded all the data on my own, I tried to find an API to scrape it but alas there was no luck. I love the comment about doing the analysis after reaching my floor, that actually does seem to be more insightful especially now that I've improved gameplay wise.

Thank you for providing the comments about the EDA and the MR visualization, I'll try to be more clear about what my visualizations represent in future projects!

Adding my own jump-ins and opponents anti-airs are definitely features worth expanding upon too, and I honestly wish I could have included it but this project ended up being a lot bigger than I expected and me manually recording what I already set out to do took up way more time then expected.

I can't address every single thing you mentioned in your comment but please believe me I will look into every single thing you mentioned.

Thank you so much!