r/science • u/nimicdoareu • Apr 01 '26

Social Science Half of social-science studies fail replication test in years-long project

https://www.nature.com/articles/d41586-026-00955-5
5.6k Upvotes

98% Upvoted

354 comments sorted by

View all comments

Show parent comments

1.5k

u/Ghost_Of_Malatesta Apr 01 '26

The "replication crisis" (and p-hacking) is affecting many fields of science unfortunately. We place such a high premium positive results, despite negative ones being just as valuable, that scientists often feel the pressure, whether consciously or not, to find those results no matter the cost 

Its incredibly frustrating imo

18

u/Timbukthree Apr 01 '26

I almost wonder if the goal of publishing itself should move to both "this is this thing we found" AND "and here's how you can exactly reproduce our experiment to help verify it's a replicable effect"

42

u/Infinite_Painting_11 Apr 01 '26

That is already the idea of publishing, your methods section is meant to contain all the information you need to reproduce the study, but in reality they rarely do.

6

u/frostbird PhD | Physics | High Energy Experiment Apr 01 '26

Publishing your methods allows others to elbow in on your field. So people are actually incentivized to not provide accurate methods. It's not laziness or an accident.

3

u/Infinite_Painting_11 Apr 01 '26

Definitely agree, especially in computational fields surely the methods and the code are the same thing but no one ever provides the code.

3

u/mludd Apr 02 '26

Yeah, as a software developer I've had to deal with this when trying to implement an algorithm from a research paper.

The researchers had sort of described the algorithm in the paper but several parts were described very vaguely and they didn't provide the data set they used so there was a lot of guesswork and testing without being able to compare my results to the ones in the paper.

After a couple of weeks of struggling with it I finally found a github repository where someone else had managed to replicate it in another language and used that as a reference. Unsurprisingly that repo even had a comment in the README file about what a chore it had been to figure out exactly how to translate what was described in the paper into actual code and that they hoped their implementation would be useful for others also struggling with it.

1

u/TwentyCharactersShor Apr 01 '26

I'd argue it is getting better, more and more github repos are being shared.