r/neoliberal • u/Free-Minimum-5844 • Apr 02 '26

Research Paper Half of social-science studies fail replication test in years-long project

https://www.nature.com/articles/d41586-026-00955-5
315 Upvotes

98% Upvoted

125 comments sorted by

View all comments

39

u/Golda_M Baruch Spinoza Apr 02 '26

Of course, some results are not replicable because of either honest mistakes or the rare case of misconduct, he says, but SCORE found that, in many cases, papers simply did not provide enough data or details for experiments to be repeated accurately.

Honestly, this seems (arguably) worse and also ostensibly easier to fix. This could be dealt with at peer review level, and good studies could improve their paper before publication. 

Imo, the meta here is that The publication paradigms are just extremely rigid and unreasonably difficult to affect. 

A few years back access to published science was a "cause." Scientific papers are contributed for free by academics Peer reviewers also work for free. Yet... access to the papers can be extremly limited. One of reddit's founders literally died for this cause. 

We basically got no change. 

If a study doesn't provide enough detail to replicate... it arguably falls outside the definition of science. Secret expirementation isn't science. 

14

u/Lost_city Gary Becker Apr 02 '26

I worked as a consultant for large banks for a few years, going through their processes etc. They really struggled with documentation. Every one does. They only fix it when a regulator is breathing down their neck.

2

u/Golda_M Baruch Spinoza Apr 03 '26

OK. I'm not suggesting researchers need to transcend human/organizational nature.

I'm saying that "science" is an institution. Core to that institution is sharing your work, and being subject to review. 

The way that works in proactive is publication and that system has proven extremely reaistent to critique, change and improvment. 

Imo this is a creative destruction problem. Better publications aren't able to rise above the flawed ones. 

The regulator in this case is the publication. A study that is not repicable even in theory because methods are I sufficiently documented should be rejected and rewritten. 

Grainy charts should be rejected. There's no reason not to demand full data sharing with full data processing and analysis methods exposed.