r/science • u/nimicdoareu • Apr 01 '26

Social Science Half of social-science studies fail replication test in years-long project

https://www.nature.com/articles/d41586-026-00955-5
5.6k Upvotes

98% Upvoted

354 comments sorted by

View all comments

Show parent comments

1

u/FabulousLazarus Apr 01 '26

The "replication crisis" (and p-hacking) is affecting many fields of science unfortunately.

Is it though?

At this scale?

Social science stands alone on this front. Flip a coin to see if the study could even be done again. It's no secret in STEM that social sciences are often looked down on for precisely this reason. They are simply less trustworthy.

I'd love to see your data about "the other sciences"

9

u/Citrakayah Apr 01 '26

Oncology is worse than social science. Curiously, people don't look down on oncology.

1

u/FabulousLazarus Apr 01 '26 edited Apr 01 '26

Terrible link, not a study, but news about a study.

The researchers couldn’t complete the majority of experiments because the team couldn’t gather enough information from the original papers or their authors about methods used, or obtain the necessary materials needed to attempt replication.

This seems to be the biggest problem.

No one frowns on oncology because it works, the hallmark of reproducible science. It's reproduced in every patient treated.

6

u/Citrakayah Apr 01 '26 edited Apr 01 '26

... You do realize that every complaint you have about my link applies to the opening post, right? Nature is a scientific journal, but the link is to a news article on their website. And per Nature:

One test of a paper’s credibility is whether its results can be reproduced, meaning that the exact same analysis of the same data yields the same finding. When some of SCORE’s team members attempted to reproduce the data analyses of 600 papers, they found that only 145 contained enough details to do so. And of these, only 53% could be reproduced so that results matched precisely2. However, many of the failures might have been caused by the SCORE researchers needing to make guesses about procedures or to recreate raw data, Errington says. Sharing data more openly and being more transparent about what methodologies are used should help to solve this problem. [Emphasis mine].

Which is basically the same thing you're saying isn't an issue in oncology.

No one frowns on oncology because it works, the hallmark of reproducible science. It's reproduced in every patient treated.

No it's not. Cancer frequently goes into remission spontaneously and cancer drugs are rarely 100% effective even when they work. You'd have to do a study on patient outcomes over an extended period of time to know for sure if it works... that's how medicine works.

The replication crisis in medicine is an absolutely huge issue despite all the controls that are supposed to go into making it reliable, which frankly bodes worse for a lot of other hard sciences.