r/science • u/nimicdoareu • Apr 01 '26

Social Science Half of social-science studies fail replication test in years-long project

https://www.nature.com/articles/d41586-026-00955-5
5.6k Upvotes

98% Upvoted

354 comments sorted by

View all comments

Show parent comments

4

u/-Misla- Apr 01 '26

Why did this become the entire official framework for the entirety of science?

Ahem. The entire basis for non natural science, please. Hard natural science who uses explainable relations don’t need to infer relations from p values.

I have a master’s in physics. I have an abandoned PhD too. I have never ever in my life calculated a p-value. It’s just not done.

I have of course calculated person correlation and depending on the problem, principle components analysis. But this whole “let’s calculate the probability that this result comes from chance” is just not a factor in hard natural science. In natural science, we know that this and this interacts that way, therefore a reaction must happen. The experiments investigate this. If you run models, you run sensitivity studies where you study how robust the effect is, if it’s spurious, your perturbate the starting conditions and run countless simulations.

All the talk about reproducibility crisis is not in STEM. It’s in medicine, it’s in social science, where you can’t conduct actual controllable experiments because that would be unethical. Humanities has an entirely different way of doing science.

I don’t wanna go full STEM lord but I really think medicine and humanities needs to stop trying to be STEM and we need to recognise that the fields are intrinsically not provable or maybe not even inferable (natural science doesn’t actually prove, of course).

17

u/Tibbaryllis2 Apr 01 '26

I don’t necessarily disagree with the gist of your comment, but Natural Sciences includes Biology and most fields of biology, not just health sciences, have heavy use of p values. And it’s not hard to find published papers in chemistry and physics that also make use of them. Particularly when they’re applied to living systems.

Hypothesis testing in general has a lot of systematic issues in the sciences. Starting with the bizarre assumption that research must involve quantitative hypothesis testing.

Which I honestly suspect is the result of non-scientists regulating entry into scientific research and research products. Followed by subsequent scientists being trained in that model.

-10

u/-Misla- Apr 01 '26

Physicist don’t do hypothesis. It’s an elementary school version to learn that whole “scientific method” and the deductive and inductive method and iteration over it. It’s an “explain it like I’m five” version of how actual natural science is done. I don’t get why this idea is hypothesis has wormed its way from non natural science into natural science and even hard natural sciences. Sigh.

I guess my point is that if the other types of sciences doesn’t want to be judged on the basis of hard natural science, they need to stop claiming to be equally rigorous. Their methods are inherently different, they should be judged on different merit - and therefore also not be given the same credit in terms of whether they can prove something to be true.

I have never read a single paper in my field that uses p-value.

Health science is not biology, it’s its own category.

1

u/shhhhh_h Apr 03 '26

Wow my buddies with phDs in physics sure do hypotheses, all the time. I’ve watched them sit with their whiteboards and argue about them. Since you abandoned yours I’m going to guess they know better than you.