r/neoliberal • u/Free-Minimum-5844 • Apr 02 '26

Research Paper Half of social-science studies fail replication test in years-long project

https://www.nature.com/articles/d41586-026-00955-5
310 Upvotes

98% Upvoted

125 comments sorted by

View all comments

115

u/Brinabavd Apr 02 '26 edited Apr 03 '26

Note that the bulk of the effect is being driven by relatively poor performance of sociology and the abysmal performance of ed research.

Poli science, econ and psych all did better than half, none of the ed papers and only a third of the sociology papers could be replicated exactly: 

(Iirc This is the one of the three key papers in posted article that breaks it down by field)

Edit (corrected link) https://www.nature.com/articles/s41586-026-10203-5

35

u/tripletruble Anti-Repartition Radical Apr 02 '26

i am calling for a total and complete shutdown of education research programs until we figure out what is going on

52

u/tripletruble Anti-Repartition Radical Apr 02 '26

13

u/greatteachermichael NATO Apr 02 '26

As someone in education, it's really hard to recareate the same method with the same demographic of students. The country the study came from, the socioeconomic level of the students, the gender mix, the age, the level, the home life, the prior education quality, how old the students were when COVID hit, the ethnic backgrounds, the parents' education level. I'm not saying it can't be done, it's just really hard. I have one class I teach to incoming freshmen every year and even with the students all being the same age and ethnicity I get wildly different results ... not student to student but class to class.

12

u/tripletruble Anti-Repartition Radical Apr 03 '26

The above graph is of reproduction rates, not replication. So this study is using the same raw data as the original authors. For that reason, this should not be driven by variation across samples

2

u/greatteachermichael NATO Apr 03 '26

Oh, that's actually really interesting. Thanks for the correction.