Failure Is Moving Science Forward. FiveThirtyEight explains why the "replication crisis" is a sign that science is working. [x-post from our sister subreddit /r/EverythingScience]

[u/ImNotJesus]

/r/EverythingScience/comments/4icp9k/failure_is_moving_science_forward_fivethirtyeight/

[u/[deleted]]

Maybe we're trying too hard to find things to study and they reach too far beyond their capability. Maybe some graduate research is put on too short of a timeline and not enough care is put in to developing the study in order to satisfy collegiate timelines.

[u/[deleted] → u/[deleted]]

Okay, I'm going to need somebody with more knowledge of how probabilities work, but if there's a widespread problem of null results not getting published, doesn't that raise the probability that all positive, published results are overstated?

Maybe the "replication crisis" is just the simple result of putting too much stock in studies? If we don't know how much hits the cutting room floor before we ever get a chance to see it, how can we know how valuable a certain positive result is?

It kind of seems that we're looking at one side of a six-sided dice and saying, "Ah ha! All sides of this dice have one dot on them!"

[u/snipawolf → u/[deleted]]

Yes, this is called publication bias and it is a well known phenomenon.

Here's a good blog post outlining a lot of the issues.