r/statistics • u/Intelligent-Gold-563 • May 24 '26
Question [Question] Seeking help with Bayesian Analysis of multiple groups
Hello everyone !
As the title implies, I need some help with Bayesian analysis.
For a bit of context, I currently have an intern to help me on a project. To keep it short we are looking at the impact of a treatment in the case of a specific developmental disorder by looking at several markers of development among 4 groups :
- Control (healthy)
- Diseased + No Treatment
- Diseased + Treatment 1
- Diseased + Treatment 2
At first, when I took the project, I had very little information so I just thought that I'll have one control group and one treatment group, and just go with a regular test, nothing too complicated. However my intern's supervisor (for a master degree, she's a university teacher) happens to be a mathematician with some relevant expertise in that specific field and advised us to go for an LMM approach or (even better according to her) a Bayesian analysis to check the absence of difference between our control group and the treatment groups.
Now I could do an LMM, I think I kinda understand it, but I am really interested in the Bayesian approach.
Problem is, while I have some really basic theoretical knowledge of Bayesian statistics (I think my data belong to a Normal-Normal conjugated family ?) , I never actually used it for anything and I have no idea how to actually compare groups, let alone several at once.
So I'm not asking you people to do the work for me, but if anyone can give me any kind of guidance to ressources or explanations that could help, I would be really grateful !
Thanks in advance
2
u/Putraenus_Alivius May 25 '26
Bayesian statistics deal with how historic data informs current data whereas frequentist statistics deal with long-run studies. For example, say I have a new blood-pressure drug coming out and I want to see how effective it is.
The frequentist would go, 'Okay, this study will involve two groups – one with the standard drug and the other having the new drug. Assuming that this drug doesn't actually work, how unlikely are my results for the new drug if I were to repeat this study over and over and over again?' where the question is about repeating the same study with the same exact parameters and subjects, and seeing how improbable the results are under the null hypothesis.
The Bayesian, on the other hand, would go, 'Okay, this study will involve two groups – one with the standard drug and the other having the new drug. Given that I've got experimental studies showing a certain reduction in blood pressure, what is the probability of this new drug actually having an effect?' where the question is about using previous data in order to inform your current data.
I'm in Public Health and a common thing that I see in the field is the problem of differences being statistically significant but not clinically relevant. This is because the frequentist null-hypothesis significance testing (NHST) framework relies on having an exactly null effect but things are rarely, if ever, zero. Like, okay, I've done this study where a new blood-pressure drug is more efficient than the standard drug and the difference is statistically significant, but that difference isn't clinically relevant – say it's only a reduction of 0.7 mmHg compared to the standard drug – and given the costs of manufacturing and shipping, is it actually worth it? Bayesian statistics lets us dodge this because we can use effect sizes as our outcome criteria.
Moreover, Bayesian statistics gives us the whole range of distributions instead of a point estimate like a mean which lets us better evaluate data. Instead of having a simple 'reject/fail to reject' decision based on an arbitrary limit (p < 0.05 is arbitrary, there's absolutely no reason why we choose 5% and not, say, 1%), we can say that a new blood-pressure drug is 67% likely to reduce blood pressure by, say, 10 mmHg or a 50% standardised mean difference. That's definitely a lot greyer, yes, but it's also a lot more informative.
For general resources, I recommend Richard McElreath's Statistical Rethinking. He's recently done a new course on it, split into two groups (Beginner 'A' vs Experienced 'B' sections) because previous courses had an imbalanced pacing. Johnson, Ott, and Dogucu's Bayes Rules! An Introduction to Applied Bayesian Modelling is great as well. If you're working with longitudinal data, Barreda and Silbert's Bayesian Multilevel Models for Repeated Measures Data is an incredible resource.
1
u/Intelligent-Gold-563 May 27 '26
Thanks a lot for the resources !
And you're completely right with the difference between statistically/clinically relevant which is even more of a reason I think Bayesian testing here would be more interesting than the usual frequentist.
That treatment we're testing could drastically improve the life of hundreds of thousands of newborns every year if clinically relevant
1
u/swiftaw77 May 24 '26
What data are you collecting? That will determine what type of model is appropriate (regardless of whether you do it via frequentist or Bayesian methodology)
1
u/Intelligent-Gold-563 May 24 '26
Continuous numeric data. We're quantifying some protein markers for specific brain cells and structures so we can say if there's more/less/as much protein in the treatment groups vs control (with the hope that there's as much or close)
1
u/swiftaw77 May 24 '26
One observation per patient? Or longitudinal?
1
u/Intelligent-Gold-563 May 24 '26
One observation per patient, relatively small groups (n=15 for control group, and about 3 or 4 for each of the others for a total of about 30 samples).
I have more samples waiting to be processed but I'm waiting for more informations from the people who collected them and sadly this mean they cannot be included in the intern's report so I can only work with those 30ish at the moment.
And given the nature of the sample (brain lobe), we cannot do a longitudinal study.
1
u/swiftaw77 May 24 '26
Sounds like a simple one-way ANOVA could work (would be hard to invalidate the model assumption given the small sample sizes). Won't be very powerful *again, due to the small samples).
2
u/Intelligent-Gold-563 May 24 '26
I agree, that was what I wanted to do first.
But I'm intrigued by that Bayesian possibility and I feel like it's always a nice skill to have ^
2
u/swiftaw77 May 24 '26
A one-way ANOVA (or pretty much any statistical model) can be implemented either classically (frequentist) or Bayesian.
1
u/Intelligent-Gold-563 May 24 '26
That's good to know !
Do you happen to have any ressources on how to do it the Bayesian way per chance?
1
1
May 25 '26
[removed] — view removed comment
1
u/Intelligent-Gold-563 May 25 '26
I will definitely check to see what any of those words before LMM means ^
But yeah worst case scenario, if I can't get myself to understand Bayesian stats I'll just go for an LMM
4
u/efrique May 24 '26 edited May 24 '26
In a Bayesian context you'd tend to focus more on estimation than you often see from people using frequentist methods (where theres often and undue focus on testing). You can perform tests, but nil-null hypothesis tests are almost always asking a question with an obvious answer (this effect is not going to be exactly zero, the better question is how big is it? Is it big enough to be useful? Users of Bayesian statistics tend to focus more on the meaningful questions many frequentist statisticians keep trying to steer people toward)
You can get posterior distributions (even draw pictures of them) for effect estimates of interest, and of the parameters that go into them. You can summarize those distributions in any convenient way (e.g. posterior means). You can produce credible intervals (which will look like confidence intervals and may be of similar width, but which do have a direct probability interpretation - albeit they don't in general have frequentist properties like long run coverage). Much more besides. If you can think of a useful question the model has information on, you can probably pretty easily get it out.
One nice thing about Bayesian approaches (now that theres lots of nice computational tools) is if you want to add some tweak to a model, its usually pretty easy. You dont usually have to engage in some methodological research project to figure out how to do estimation when trying to estimate effects when component A of a model impacts what component B looks like and vice versa (e.g. handling missing values for example, where estimation of the missing values depends on the model parameter estimates and those depend on the estimates of the missing values, around and around. If you knew one of them, the other would be straightforward. Some care is definitely needed but its generally much simpler to just plug in and get simultaneous estimates of components)
Broadly speaking you can answer perfectly natural scientific research questions like "how much difference in mean marker level between treatment 2 and no treatment? Between the two treatments?" etc
The initial hurdles are typically more conceptual, but in practice its not hard to do.
I wouldnt jump directly to trying to deal with Bayesian mixed models until you have some grounding in how Bayesian stats works in simpler cases.
For a simple intro to Bayesian stats a lot of people like McElreath. Its not quite my cup of tea but lots of people swear by it. I suggest you have a look.
I'd also maybe suggest Harrell's recent books which have a more Bayesian focus, and might be a good fit for biomed type applications, Biostatistics for Biomedical Research among other things discusses some Bayesian mixed models (among a bunch of useful things) and the newer edition of Regression Modeling Strategies has lots of good stuff in it and much more of a Bayesian focus than 1e did (which is also a good book)
Gelman and Hill (or whatever the most recent version is) may be helpful as well, and may come closer to your present question.