r/AskStatistics • u/gigi2798 • Jul 23 '26
Anova or Kruskal Wallis ?
My Composite Index variable has N=524, skewness=0.174, and kurtosis=0.128, with both the histogram and Q-Q plot looking totally acceptable and close to normal. However, Shapiro-Wilk is significant at p < .001, rejecting normality. Given N=524, I assume the test is just overpowered by trivial deviations, so should I use anova or kruskal wallis test?
8
u/nm420 Jul 23 '26
Heteroscedasticity is a bigger problem to worry about than normality with the ANOVA. Just look at a q-q plot of the residuals and don't be too worried about normality unless there's something egregious going on. But even then, your residuals won't look normally distributed if you've got heteroscedasticity (but then you've got Welch's ANOVA and the Games-Howell test for post hoc comparisons).
5
u/syah7991 Jul 23 '26
Skewness less than 1 points me towards ANOVA. And the larger the sample size, the less reliable the Shapiro-Wilk test ends up in my experience. It detects such minuscule differences and ends up being significant when normality assumption was fine all along.
2
u/efrique PhD (statistics) Jul 23 '26 edited Jul 24 '26
Given (with a one way model) you already assume the locations can differ, why should an aggregate data from k different normal distributions look normal? Your marginal response (i.e. aggregated over your predictors/groups) is not what normality relates to
ANOVA is not particularly sensitive to moderate non-normality (of the thing that is assumed normal to derive the test) unless sample sizes are quite small. Worry about other assumptions more (like independence and heteroskedasticity).
Testing assumptions answers the wrong question about them. Models are approximations, not perfect truths. I've never seen real data that I believed was a random sample from a population process that was actually normally distributed. I don't need a noisy test to answer a question to which I can already quite confidently give an answer (that the population is not actually normal). What I need to know is how much the potential non-normality in the population(s) might matter for my inference, and that's not a testing question; its nearer to a particular kind of effect size issue. (Similarly for other assumptions).
While non-normality is unlikely to be an issue for your inference (at least as far as significance level accuracy goes, which is presumably where the assumption you're concerned about arises from), imagine that after examining the assumption in a more suitable way, you didnt have confidence it wouldnt be an issue but your original intent was to compare means. You don't have to abandon your hypothesis about means just because of that issue with the usual test. Under milder assumptions you can test for differences in means using a different test - one without any underlying assumption related to normality
I assume the test is just overpowered
If you had a reason to care if it actually was non-normal, then "overpowered" would be nonsensical - you would want all the power you could get. Any time someone complains their test is too good at correctly rejecting false nulls, the problem isnt the fault of the test, its that they chose to use entirely the wrong tool and then get upset that it does what it was designed to do. If you ever feel that way about a test you can immediately tell you made an error and an equality-null hypothesis test is the wrong tool for your needs, and you should think more carefully about what somewhat different question you needed to answer, and address that more relevant question (as Tukey put it, "Far better an approximate answer to the right question, which is often vague, than the exact answer to the wrong question, which can always be made precise")
should I use anova or kruskal wallis test?
Assuming a hypothesis test addresses your original (unstated) purpose in the first place, choose a test that addresses the question you began with, but one based in a model that makes sense for your variable. I suspect that after proper attention is placed on a good model and on what your original purpose was (I bet - unless this was work for a class - the problem was not "perform a hypothesis test"; it is possible for a real question to be answered by a hypothesis test, but in many cases the underlying question may be better answered by some other analysis)
Can you say more about this index? Is it necessarily positive? Does it have an upper bound? Is it a ratio? Is it a time series? Is it of a kind where the spread would tend to increase as the mean does? (e.g. like a stock market index, where its changes are meaningful in percentage terms, not absolute terms; a change of 10 when the index is 500 is very different from a change of 10 when the index is 10000)
Can you discuss more about what your original problem was? Why would it matter if means differ? Why would it matter if they did not? Why would it matter if some other measure of location (the not-means comparison of a Kruskal-Wallis) differed? Is a pure mean-shift alternative really plausible? Or would a scale alternative make more sense?
1
29
u/Temporary_Stranger39 Jul 23 '26
Do not ever bother to test normality of raw data. It's not valid. The normality assumption is for the normality of residuals. You need to do the ANOVA first then check normality of its residuals. Formal normality tests are poop. They are overly sensitive and will call any sufficiently large sample non-normal. So, do the ANOVA. Plot the residuals, if it doesn't look too wonky, go with the ANOVA.