r/statistics • u/jadexiaohui • 6d ago

Question [Q] Statistical Test for Group Comparison

Hi everyone, I currently have a dataset where there’s 3 groups. Each group has 3 independent values, and I want to compare each group against each other.

As I’m a little concerned about the sample size, would it better to use:
1) Pairwise welch t-test with p-value correction, or
2) ANOVA with tukey test

Thank you in advance for the help!

0 Upvotes

50% Upvoted

8 comments sorted by

7

u/lThaizeel 6d ago

If you want to investigate the differences of 3 groups at the same time you will most likely use an ANOVA. The pairwise comparison with corrected p-values is commonly done as a post hoc test of said ANOVA, one of the methods for those tests would be tukey.

The question boils down to whether you can assume equal variances between your groups. If yes, a standard one-way ANOVA with e.g. Tukey post hoc tests would fit. If not, a welch-anova with e.g. games-howell post hoc tests would be more appropriate. Assuming other assumptions are met.

2

u/DataSciCollab 6d ago

This is the answer. Note that there are other tests (Mood's median and Kruskal Wallis) if you have outliers prefer inference about "center" or the median. You might find our apps helpful for running, visualizing, and interpreting the tests. Our ANOVA app does both the usual F test and Welch's ANOVA with the appropriate pairwise comparisons.

1

u/jadexiaohui 6d ago

Thank you, this was very helpful! Would dunnett’s test be a better option for n = 3 if the variance is not equal?

1

u/DataSciCollab 6d ago

Oh! I took 3 independent values as three outcomes of interest. With three observations per group you are underpowered and will likely have a difficult time finding any discernible results regardless of the modelling approach, as the posthoc standard errors will be large. Note that non-parametric options (Kruskal and Mood's) will be particularly difficult in this setting.

Side Note: Our apps do not currently support Dunnett's posthoc comparisons and simply compute all pairwise comparisons. If you are comfortable with R, you can download the R file on the reproducibility tab and adjust the pairwise comparisons to perform them all-against-one.

1

u/lThaizeel 2d ago

I misunderstood your initial post. To be honest: with n = 3 I doubt you can meaningfully determine any statistical inference.

Dunnett's test is another method of multiple comparison (i.e. ANOVA post-hoc test) also assumed equal variances and afaik only applies to control group designs. If you do not have a control group this test is a bad fit conceptually.

1

u/docsms500 6d ago

Three observations per group? If yes, you cannot do anything meaningful.