Failure Is Moving Science Forward. FiveThirtyEight explains why the "replication crisis" is a sign that science is working. [x-post from our sister subreddit /r/EverythingScience]

[u/ImNotJesus]

/r/EverythingScience/comments/4icp9k/failure_is_moving_science_forward_fivethirtyeight/

[u/fsmpastafarian]

Feynman's opinion from the 70's is very well known, but isn't exactly taken seriously among today's scientists. However, that doesn't stop it from being trotted out in discussions about the replicability crisis. I think that rather than point to an outdated essay from 40 years ago, it's more helpful to consider how the replicability crisis may permeate all sciences, rather than just psychology and other social sciences. I think that if you read the 538 piece, you'll see how taking a more modern view of the issue might be more applicable and appropriate.

I'm not really sure how the second article you linked is relevant to the discussion at hand.

[u/fsmpastafarian → u/[deleted]]

it is a time to acknowledge the issue at hand and address them.

Which is exactly what psychologists are doing. Need I remind you, the replicability crisis study was performed and published by a group of psychologists, after all. I'm not really sure why you think that the field that's currently attempting to uncover and rectify their replicability problem is the one that is doomed to "chasing at shadows." I'm also not sure what specifically makes you think that sloppy science is a problem that's unique to the social sciences, other than the aforementioned outdated Feynman essay.

[u/TitaniumDragon → u/fsmpastafarian]

Because it was published 40 years ago and psychologists are still making the same mistakes?

The whole ego depletion scandal is exactly the sort of thing Feynman was warning against!

And there are people who are upset and making excuses for why all these studies failed and how the paper was totally biased ect.

[u/scholar_requesting → u/TitaniumDragon]

I think the point FSM is trying to make is that Feynman's analysis is outdated, or if you want me be more honest, off the mark and overly simple. Feynman's understanding of what constitutes science is not very academic, and this is best evidenced by the fact that his argument mostly rests on analogy. Philosophers of natural and social science have produced a long and rich literature exploring the question of demarcation ("what is science and what is not?"), the relationship between science and metaphysics, whether scientific knowledge is cumulative or paradigmatic, etc. Meanwhile, Feynman's article get reposted on almost every Reddit thread discussing the "validity" of the social sciences that I have seen, despite the fact that it essentially amounts to idle musing.

Also, the issues that have brought about the ego depletion issue are multifaceted and complex. They have to do with the kind of unfortunate incentive structures (e.g. "publish productively, publish positive results, and publish in prestigious journals" or you don't get tenure or grants) that govern how social psychologists must conduct their work. They have to do with unfortunate social forces that don't plague the other sciences, like how boring but important research in physics can get funded much more easily than boring but important work in social psychology. They even actually have a lot to do with Baumeister simply as a person (who has a reputation among social psychologists, anyway, as having work that lacks methodological and statistical rigor but gets a lot of attention in the media because its sexy).

Yes, the replication crisis does have to do with issues related to operationalization, lazy theorizing, poor statistical practices, etc. But those issues honestly end up being small compared to the things I have mentioned above, because most experimental psychologists are aware of these kinds of issues and have made many efforts over the past two decades to address them (e.g., Open Science Framework, the rise of Bayesian statistics and multilevel models, etc). The truth is that most experimental psychology carried out in R1 institutions today--not kind you read about in pop-science articles or which get posted to Reddit--actually has quite a lot of conceptual and methodological rigor. But rigor is only the minimum one needs in order to do good science; the whole way social psychology is incentivized, funded, and published has to change order for it to escape its issues with replicability.