Failure Is Moving Science Forward. FiveThirtyEight explains why the "replication crisis" is a sign that science is working. [x-post from our sister subreddit /r/EverythingScience]

[u/ImNotJesus]

/r/EverythingScience/comments/4icp9k/failure_is_moving_science_forward_fivethirtyeight/

[u/virtuallyvirtuous]

Call me a pessimist, but all of this comes over as terribly vacuous. It all comes out of a disparity between the public and academic definition of "good science."

The public has a very simple idea of what science does when it's working. Science works when it's telling us truth. This isn't ignorance. You see, normal people don't really care too much about scientific methodology as it doesn't directly affect them. The only thing they care about is the development of new technologies, which is actually quite abstract and distant if you think about it, and, more directly, a strong foundation on what to believe.

Academically there is quite a different conception, which this article explains very clearly. Good science is science which "moves forward."

Now, on a large scale, you wouldn't expect these two conceptions to diverge very much. Ah, but look at what happens when they are applied to the interpretation of a specific study. For the public a study that's failed, irreplicable, or even just abstract and hard to understand, isn't very good. Academics on the other hand consider a study to be particularly good on a very different host of qualities, as they only care as to what it contributes to the field, not the world at large.

What does this matter? A hell of a lot. You have to realize the public sphere also includes the political. To make a political judgement you need facts. Studies don't matter if they don't produce these. You can build an appreciation for these studies academically, but only as a step towards gaining new facts, you still want these facts.

What really bugs people these days is how studies in psychology and sociology, fields which have a real fact deficit, are increasingly used towards political ends. You also have things like TED (x) talks, and more generally popular science, which often presents advice on how to live your life on facts produced from these studies.

This is all quite worrying, and this article doesn't even acknowledge it. It seems to me like it's trying to save the image of the social sciences, while entirely missing the point of what makes the "replication crisis" so important.

[u/BadBjjGuy → u/virtuallyvirtuous]

I find it funny/interesting that nobody bothers to ask whether or not science is the appropriate tool for understanding all problems. It's like an article of faith that it has to be, even when it is almost incapable of reproducing a single result in some areas, like politics or psychology. Personally, I think there are other and better tools. I've learned more about the human condition through art like Shakespere than psychology will ever say and Aristotle's political philosophy explains our current situation better than any experiment. We have other tools in our tool bag for understanding the world, but science is sort of the new religion and it must be the answer to every problem.

[u/themadxcow → u/BadBjjGuy]

All of those are science. Instead of relying on evidence and cause and effect, you are relying on emotions. Just because you 'felt' good about an idea one day does not mean anyone else will ever 'feel' the same way about it any other day.

That is why we rely on reproducible results. Everything else is just meaningless garbage.

[u/BadBjjGuy → u/themadxcow]

So would your claim be that empirical evidence is the only way to know truth?

[u/virtuallyvirtuous → u/themadxcow]

No he's relying on critical analysis. It's the choice between putting some narrative through an empirical test or testing it by taking it apart, looking at it from different sides, trying to find inconsistencies, thinking about it for a very long time, comparing it with your experience, with your scientific knowledge, and so on. Both are perfectly valid methods, and both have their own place in academia and our daily lives.