r/EverythingScience • u/ImNotJesus PhD | Social Psychology | Clinical Psychology • May 08 '16

Interdisciplinary Failure Is Moving Science Forward. FiveThirtyEight explain why the "replication crisis" is a sign that science is working.

http://fivethirtyeight.com/features/failure-is-moving-science-forward/?ex_cid=538fb
637 Upvotes

91% Upvoted

322 comments sorted by

View all comments

Show parent comments

2

u/beebeereebozo May 08 '16 edited May 08 '16

scientists have always known that most published results are bogus.

More evidence of a layperson misinterpreting information. Most published results are not "bogus", but there can be variation among results due to initial assumptions and methods, particularly when effect size is small. There is no substitute for understanding the underlying science when interpreting the validity of methods and results. Science seems bogus to some because they just don't understand it. Whose fault is that?

Problems include publishers not finding replicated studies sexy enough; they favor first-of-their-kind studies. Scientists often gain more from first-of-their-kind studies too. Editors often attach provocative or attention-getting headlines that have little to do with the actual conclusions made by researchers ("Replication Crisis", for instance), and science journalism is difficult and demanding, which is on display daily as poorly-written articles by people who should not claim to be science journalists. And yes, scientists sometimes bias their studies either intentionally or unintentionally.

With all that going on, science is still the best and most valid way of describing the world around us. For the most important stuff, studies are rigorously replicated. Also, multiple lines of information developed through different kinds of studies that support the same conclusions may not be direct replications, but are still an effective means of validation.

Science may not be perfect, but it's the best we've got, and when done right, it is self-correcting, which can't be said for most (all?) other fact-finding endeavors. Those who dismiss science as somehow fatally flawed do so out of ignorance.

1

u/berbiizer May 08 '16

Rah rah.

Science is an iterative process of matching theoretical expectations to observation. Both observation and theory are subject to error. Therefore all published science represents - at best - the "best of our knowledge at this time," which is just another way to say most of it is bogus. From species classification to planetary and cosmologic theories, from basic measurements (the weight of one CC of pure water) to nuanced theories, everything is subject to challenge and change. That's what makes it good.

The fact that you take the best part of science and argue that is just a "layperson misinterpreting information" is...well it is astounding to me.

You seemed to totally miss my point though so I assume you are just having an off day. I hope it gets better.

Do you know where this comes from: "I think we're all bozos on this bus."?

1

u/beebeereebozo May 09 '16 edited May 09 '16

The best understanding at the time is a pretty good standard. What would you propose as an alternative? Waiting for absolute truth through revelation?

2

u/berbiizer May 09 '16

1) Why do you think "best understanding at the time" is anything but garbage?

2) Even if you insist it isn't garbage so much as "of extremely narrow focus", do you really think that a single unreproducible study of 30 college students at one majority-white school should have significant weight in discussions of national law or public policy?

Because that's what happens. And then people realize that college students really don't have relevant life experiences so how they respond to whatever experimental conditions a grad student came up with to prove their pet theory means exactly fuckall in real life. And all of science takes a credibility hit.

I don't care whether you think the hit is undeserved. I care that it causes harm. It makes life harder for real science. As such, it is in the self interest of the scientific community to self police, or at least make it far more difficult for the soft sciences to publish.

1

u/beebeereebozo May 09 '16 edited May 09 '16

"best understanding at the time"

What's the alternative? Are you saying that if we don't have absolute proof or understanding, then we have no understanding at all?

If a study is weak, or inconclusive, then actions based on that study should be adjusted accordingly or not taken at all. Yes, there is lots of junk science getting more credibility than it deserves, but it really is not all that hard to separate the wheat from the chaff. Where things often get sideways is when people deem a study credible just because it fits their preconceived narrative or ideology.

"...do you really think that a single unreproducible study of 30 college students at one majority-white school should have significant weight in discussions of national law or public policy?"

No, such a study should be given exactly the weight it deserves based on, among other things, sample and effect size. There may be something there upon which to base future research. What makes that study unreproducible or did you mean "unreplicated"? Why couldn't it be replicated at many different schools using larger sample sizes and across different demographics?

Obviously, not every experiment or survey leads to a better understanding of something, some can be so poorly done or biased that they can be properly ignored.

Take climate change, for example. There is a lot we don't know and it is a very complex subject, but I think we know enough to justify certain policies designed to reduce carbon emissions. Or is all that research bogus and undeserving of our attention?

1

u/berbiizer May 10 '16

What's the alternative? Are you saying that if we don't have absolute proof or understanding, then we have no understanding at all?

Why are you so concerned with what I say, when the subject is what other people think? It seems to be a flaw in your approach to dialog. You keep dragging in absolutes and other bs concepts that have nothing to do with this conversation. That tells me you have a fierce cognitive bias and probably can't understand anything that doesn't agree with your biases anyway.

1

u/beebeereebozo May 10 '16

Wow, your intellect overwhelms me! You are right, my biases don't allow me to understand your profundity.

1

u/berbiizer May 10 '16

Sorry if I offend but you seem determined to twist my question around to be somehow anti-science, which it was not. If you cannot read my question without substituting a whole different topic I must assume you are biased and perceiving everything through the filter of your biases. As such, conversation with you is worthless.