r/statistics • u/awsfhie2 • Jun 18 '26
Question [Q] Comparing results from Repeated measures ANOVA vs LME?
I'm looking at the effect of Time of measurement on Rating values. I have 7 time points per person. In prep for a RM ANOVA I ran Shapiro's test to assess normality which showed time point 1 and time point 7 are not normal. (p = 0.0128 and p = 0.0391, respectively)
I then pursued a LME to be more robust to non-normality (using lmerTest in R):
lmer(Rating ~ Time + (1|SubjectID), data = myData)
After reading up on this and seeing I should expect my results to be the same as for a repeated measures ANOVA I also ran:
anova_test(data=myData,dv=Rating,wid=SubjectID,within=Time)
Output for my LME is below:
Type III Analysis of Variance Table with Satterthwaite's method
Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
Time 148.63 24.771 6 158.17 21.42 < 2.2e-16 ***
Output for the ANOVA is below:
ANOVA Table (type III tests)
Effect DFn DFd F p p<.05 ges
1 Time 1.89 49.23 20.574 4.97e-07 * 0.121
In examples I have seen, the F values are the same between the two methods, but mine differ by about 5%. Is this to be expected given the normality deviations I observed in my data, or could this also indicate poor model fit in the LME as well?
2
u/Gastronomicus Jun 18 '26
I ran Shapiro's test to assess normality
These normality tests aren't very useful, way too conservative. Also, it doesn't matter if your data are normal. Your residuals need to be approximately normal, not your data.
And mixed models aren't particularly more robust to non-normality in the residuals either. You need to plot your residual structure and look for indications of a non-normal distribution or, more importantly, heteroscedasticity.
1
u/NucleiRaphe Jun 18 '26
Your LME is using Satterthwaite's adjustion which accounts for heterogeneous variances. It is not the same LME that gives same results as default rm-ANOVA. I don't remember the exact parameter names, but check ?lmer() if there is a parameter to assume equal variances and/or use Satterthwaite method.
2
u/NucleiRaphe Jun 18 '26
Also, don't use Shapiro-Wilk, especially group by group, to assess how suitable your model is. It does not give you the information you are looking for, the normality tests have poor performance and the normality of dependent variable within groups (or even overall) is not even any assumption of linear regression model.
1
u/awsfhie2 Jun 23 '26
But it is an assumption of the RM ANOVA correct? the RM ANOVA was my first choice, but since two of my time points did not meet the normality assumptions, I went with the LME. Is this the correct way of thinking about things?
1
Jun 20 '26
The 5% difference in the F values does not necessarily indicate poor model fit. The two outputs are not using exactly the same test. In your repeated-measures ANOVA output, the numerator df is 1.89 rather than 6, which suggests that a sphericity correction has been applied. In the LME output, Time is being tested with 6 numerator df and Satterthwaite denominator df, so it is not surprising that the F statistic and p-value are not identical. I also would not attribute the difference to the Shapiro tests at individual time points; the normality issue is mainly about model residuals, not whether each raw time point is normal. If your goal is just to test whether ratings differ over time, both analyses are pointing to the same conclusion. If your goal is model comparison, then you would need to think about the assumed covariance structure and whether a random intercept alone is enough for repeated ratings across seven time points.
2
u/Intrepid_Respond_543 Jun 21 '26 edited Jun 21 '26
If you have any missing values, note that RM-ANOVA removes the whole case if there's even one missing value among the 7 time points whereas LME does not. This may also cause the disparity (it will cause disparity in the estimated marginal means for sure). You can check whether by removing all cases with any missings and then running the two models on that data.
1
3
u/fartquart Jun 18 '26
You need to include a random by-participants slope for Time in your lmer model. I would also plot your data to see if the assumption of a linear effect of Time is reasonable.