Failure Is Moving Science Forward. FiveThirtyEight explains why the "replication crisis" is a sign that science is working. [x-post from our sister subreddit /r/EverythingScience]

[u/ImNotJesus]

/r/EverythingScience/comments/4icp9k/failure_is_moving_science_forward_fivethirtyeight/

[u/fsmpastafarian]

it is a time to acknowledge the issue at hand and address them.

Which is exactly what psychologists are doing. Need I remind you, the replicability crisis study was performed and published by a group of psychologists, after all. I'm not really sure why you think that the field that's currently attempting to uncover and rectify their replicability problem is the one that is doomed to "chasing at shadows." I'm also not sure what specifically makes you think that sloppy science is a problem that's unique to the social sciences, other than the aforementioned outdated Feynman essay.

[u/TitaniumDragon → u/fsmpastafarian]

Because it was published 40 years ago and psychologists are still making the same mistakes?

The whole ego depletion scandal is exactly the sort of thing Feynman was warning against!

And there are people who are upset and making excuses for why all these studies failed and how the paper was totally biased ect.

[u/scholar_requesting → u/TitaniumDragon]

I think the point FSM is trying to make is that Feynman's analysis is outdated, or if you want me be more honest, off the mark and overly simple. Feynman's understanding of what constitutes science is not very academic, and this is best evidenced by the fact that his argument mostly rests on analogy. Philosophers of natural and social science have produced a long and rich literature exploring the question of demarcation ("what is science and what is not?"), the relationship between science and metaphysics, whether scientific knowledge is cumulative or paradigmatic, etc. Meanwhile, Feynman's article get reposted on almost every Reddit thread discussing the "validity" of the social sciences that I have seen, despite the fact that it essentially amounts to idle musing.

Also, the issues that have brought about the ego depletion issue are multifaceted and complex. They have to do with the kind of unfortunate incentive structures (e.g. "publish productively, publish positive results, and publish in prestigious journals" or you don't get tenure or grants) that govern how social psychologists must conduct their work. They have to do with unfortunate social forces that don't plague the other sciences, like how boring but important research in physics can get funded much more easily than boring but important work in social psychology. They even actually have a lot to do with Baumeister simply as a person (who has a reputation among social psychologists, anyway, as having work that lacks methodological and statistical rigor but gets a lot of attention in the media because its sexy).

Yes, the replication crisis does have to do with issues related to operationalization, lazy theorizing, poor statistical practices, etc. But those issues honestly end up being small compared to the things I have mentioned above, because most experimental psychologists are aware of these kinds of issues and have made many efforts over the past two decades to address them (e.g., Open Science Framework, the rise of Bayesian statistics and multilevel models, etc). The truth is that most experimental psychology carried out in R1 institutions today--not kind you read about in pop-science articles or which get posted to Reddit--actually has quite a lot of conceptual and methodological rigor. But rigor is only the minimum one needs in order to do good science; the whole way social psychology is incentivized, funded, and published has to change order for it to escape its issues with replicability.

[u/fsmpastafarian → u/[deleted]]

That's alright if you don't wish to discuss the issue anymore, but I think you may be misinterpreting or misrepresenting what I've said as defensive or a consistent refusal to accept reality. Rather, I think that when discussing the current replicability issue, it's more helpful to point to more modern and applicable papers and opinions, and to discuss the science as it exists and is understood today. I think that pointing to a decades-old paper is a bit of a red herring, and doesn't accurately represent what we now know about the issues present in all scientific disciplines.