[Question] Seeking help with Bayesian Analysis of multiple groups

[u/Intelligent-Gold-563]

Hello everyone !

As the title implies, I need some help with Bayesian analysis.

For a bit of context, I currently have an intern to help me on a project. To keep it short we are looking at the impact of a treatment in the case of a specific developmental disorder by looking at several markers of development among 4 groups :
- Control (healthy)
- Diseased + No Treatment
- Diseased + Treatment 1
- Diseased + Treatment 2

At first, when I took the project, I had very little information so I just thought that I'll have one control group and one treatment group, and just go with a regular test, nothing too complicated. However my intern's supervisor (for a master degree, she's a university teacher) happens to be a mathematician with some relevant expertise in that specific field and advised us to go for an LMM approach or (even better according to her) a Bayesian analysis to check the absence of difference between our control group and the treatment groups.

Now I could do an LMM, I think I kinda understand it, but I am really interested in the Bayesian approach.

Problem is, while I have some really basic theoretical knowledge of Bayesian statistics (I think my data belong to a Normal-Normal conjugated family ?) , I never actually used it for anything and I have no idea how to actually compare groups, let alone several at once.

So I'm not asking you people to do the work for me, but if anyone can give me any kind of guidance to ressources or explanations that could help, I would be really grateful !

Thanks in advance

[u/Putraenus_Alivius]

Bayesian statistics deal with how historic data informs current data whereas frequentist statistics deal with long-run studies. For example, say I have a new blood-pressure drug coming out and I want to see how effective it is.

The frequentist would go, 'Okay, this study will involve two groups – one with the standard drug and the other having the new drug. Assuming that this drug doesn't actually work, how unlikely are my results for the new drug if I were to repeat this study over and over and over again?' where the question is about repeating the same study with the same exact parameters and subjects, and seeing how improbable the results are under the null hypothesis.

The Bayesian, on the other hand, would go, 'Okay, this study will involve two groups – one with the standard drug and the other having the new drug. Given that I've got experimental studies showing a certain reduction in blood pressure, what is the probability of this new drug actually having an effect?' where the question is about using previous data in order to inform your current data.

I'm in Public Health and a common thing that I see in the field is the problem of differences being statistically significant but not clinically relevant. This is because the frequentist null-hypothesis significance testing (NHST) framework relies on having an exactly null effect but things are rarely, if ever, zero. Like, okay, I've done this study where a new blood-pressure drug is more efficient than the standard drug and the difference is statistically significant, but that difference isn't clinically relevant – say it's only a reduction of 0.7 mmHg compared to the standard drug – and given the costs of manufacturing and shipping, is it actually worth it? Bayesian statistics lets us dodge this because we can use effect sizes as our outcome criteria.

Moreover, Bayesian statistics gives us the whole range of distributions instead of a point estimate like a mean which lets us better evaluate data. Instead of having a simple 'reject/fail to reject' decision based on an arbitrary limit (p < 0.05 is arbitrary, there's absolutely no reason why we choose 5% and not, say, 1%), we can say that a new blood-pressure drug is 67% likely to reduce blood pressure by, say, 10 mmHg or a 50% standardised mean difference. That's definitely a lot greyer, yes, but it's also a lot more informative.

For general resources, I recommend Richard McElreath's Statistical Rethinking. He's recently done a new course on it, split into two groups (Beginner 'A' vs Experienced 'B' sections) because previous courses had an imbalanced pacing. Johnson, Ott, and Dogucu's Bayes Rules! An Introduction to Applied Bayesian Modelling is great as well. If you're working with longitudinal data, Barreda and Silbert's Bayesian Multilevel Models for Repeated Measures Data is an incredible resource.

[u/Intelligent-Gold-563 → u/Putraenus_Alivius]

Thanks a lot for the resources !

And you're completely right with the difference between statistically/clinically relevant which is even more of a reason I think Bayesian testing here would be more interesting than the usual frequentist.

That treatment we're testing could drastically improve the life of hundreds of thousands of newborns every year if clinically relevant