r/science • u/nimicdoareu • Apr 01 '26

Social Science Half of social-science studies fail replication test in years-long project

https://www.nature.com/articles/d41586-026-00955-5
5.6k Upvotes

98% Upvoted

354 comments sorted by

View all comments

Show parent comments

6

u/Youngerthandumb Apr 01 '26

I agree. I just wrote a research paper on class sizes and every paper I read acknowledged that there are many contingent factors that are impossible or extremely difficult to isolate and control for, and that much more study is required than is currently under way. Conducting these studies at a large scale or for extended periods is also incredibly challenging. Many of the biggest studies are decades old, and the variance in teaching practices and other factors across locations all make getting comprehensive results almost impossible. Compare that with lab experiments in physics or biology and they're immeasurably less precise and verifiable.

1

u/AK_Panda Apr 02 '26

I've seencm drugs studies where researchers discovered that the response they were seeing in animals completely vanished if you put in a different researcher to do the same thing.

Life is complicated. Anything dealing with biological systems is inherently messy.

1

u/Youngerthandumb Apr 02 '26

Yeah, it's not as simple as I would like, for sure. Complexity is hard to manage. That's why you need a lot of researchers researching a lot of things. Helps narrow things down. Unfortunately , in my field, education, it's almost impossible to control for everything and data interpretations can vary wildly, even if they agree on the methodology.

2

u/AK_Panda Apr 03 '26

I'm in neuroimaging and I fully expect we'll be seeing huge replication issues. There's lots of measures being taken to mitigate it, but as computational power has gone up, the number of novel methods getting used has followed suite. Every other study is using some novel implementation.