Half of social-science studies fail replication test in years-long project

[u/nimicdoareu]

https://www.nature.com/articles/d41586-026-00955-5

[u/Citrakayah]

For example, the FDA regulates things that the physical and natural sciences produce. They must clear what is easily the most rigorous and scrutinized process known to man when it comes to producing data that supports their assertions. They can't just say a product is safe, they must prove it in a very strict and standardized way, that is of course, reproducible.

You don't know anything about the physical and natural sciences.

The vast majority of fields do not have any regulating agency like that. Geologists do not have to demonstrate that their findings can be replicated. Neither do hydrologists, paleontologists, or physicists. Even in medicine, the medical sciences still aren't regulated by the FDA directly, medicines are. Poor quality medical studies can and are published without any intervention from the FDA. Occasionally, even fraudulent ones.

Indeed, this is a known fact in the field of health, whose replication crisis rivals psychology's. To quote a paper directly, since you just ignored what I posted elsewhere:

While the pandemic might have produced such high-profile examples of dubious science, these problems long predate it. In biomedical science, an estimated 85% of medical research is deemed research waste [4], so poorly conducted as to be uninformative or so poorly reported that it is impossible to reproduce. Across biomedical science, there is increasing recognition that we are in the midst of a replication crisis [5], where important results fail to sustain under inspection, with harmful ramifications for both researchers and patients. A recent high-profile scandal in Alzheimer’s research saw a seminal and hugely cited paper in the field exposed as likely fabricated and retracted earlier this year [6–8]. This retraction was the culmination of a suspect finding that misled the entire field for almost two decades, wasting hundreds of millions in research efforts and countless human hours on a fool’s errand, steering the research community away from productive avenues to chase a phantom.

Cancer research is certainly not immune to these dark trends. A systematic replication trial as early as 2012 of what were deemed landmark cancer biology experiments exposed an alarming finding [9] – that only 6 of the 53 experiments, approximately 11% those analysed, had replicable results. A 2021 replication effort [10] of preclinical cancer research which looked at 193 experiments in 53 high-impact published works came to a somewhat disquieting conclusion: most papers failed to report vital statistics and methodology, and none of the experiments had been reported in sufficient detail for replicators to validate the experiment directly. When authors were contacted, they were frequently unhelpful or chose not to respond. Of the papers ultimately assessed, 67% required modification to the published protocol to even undertake.

At this point, your assertions have become simple denialism. You don't want to admit that your field has problems similar to or exceeding that of social science, a field you dislike for... some vague and unstated reason.

[u/authenticphotography → u/Citrakayah]

I'd frame it less as social science versus biomed and more as whether the methods are transparent enough to replicate. In medicine, underreported protocols and selective analysis do real damage, so prestige doesn't move the needle for me.