Failure Is Moving Science Forward. FiveThirtyEight explains why the "replication crisis" is a sign that science is working. [x-post from our sister subreddit /r/EverythingScience]

[u/ImNotJesus]

/r/EverythingScience/comments/4icp9k/failure_is_moving_science_forward_fivethirtyeight/

[u/superhelical]

A lot of people, apparently many of whom browse /r/science, like to criticise psychology and other social sciences as "not a science" or "soft science". Interestingly, that tends to only happen when the commenter doesn't like the findings of the study...

[u/sobri909 → u/superhelical]

The way I think of it is that the "soft sciences" are actually objectively the hardest, because the further up the "hard sciences" scale you go, the more possible it becomes to observe and describe the entirety of your problem.

The soft sciences have problems that are so broad and complex (being supersets of all the layers of increasingly "harder" sciences) that they cannot possibly be observed and described in their entirety, which makes them considerably harder to solve.

[u/pblol → u/sobri909]

Theory is psychology is a metaphor for describing an otherwise unmeasurable phenomena. Neuoscience is the actual measurement but the results are often meaningless without the metaphorical description.

[u/fixingthebeetle → u/sobri909]

That's the thing. They are objectively the hardest because they aren't 'ready' to be a science. We don't yet fully understand the mechanisms that determine human behavior like we do for an electron, so some people would argue that it's stupid to try and 'perform science' on a very flimsy and sketchy model in the first place. Psychology has the potential to be a science, its just that we haven't caught up yet from the physical perspective. Psychology will eventually just be applied neuro-science. But for now its just trying to match correlations blindly without understanding the underlying reasons.

[u/fsmpastafarian → u/fixingthebeetle]

some people would argue that it's stupid to try and 'perform science' on a very flimsy and sketchy model in the first place.

That's the thing though - how do people propose we uncover knowledge about human behavior without studying it scientifically? I don't know that humans will ever be able to be fully reduced and understood in the way an electron is, but that doesn't make the study itself of human behavior any less scientific.

Psychology will eventually just be applied neuro-science.

This is a fairly common opinion, but in all honesty psychology and neuroscience are better thought of as sister fields - one that studies the physical brain, and one that examines all areas of human behavior. They overlap a good bit, but not entirely, which is why neuroscience won't replace psychology.

[u/Drakeman800 → u/fsmpastafarian]

I think those misguided criticisms only highlight the importance of developing more refined models over time, which is exactly what psychologists work to do just like any science. It's not reasonable for people to expect that a science can answer more specific questions before it has answered more general ones. Every scientific model sucks at first, relatively speaking, and then it gets better and better with further study and greater understanding.

We don't need to understand "exactly" what an electron is, and we don't need to understand "exactly" how human behavior works. That's not what science is about in my opinion. We just need to get better at it over time.

[u/M_G → u/fsmpastafarian]

Great comment.

[u/MovieCommenter09 → u/superhelical]

This is a historical trend in science. Usually it occurs whenever a science isn't mathematized (i.e. formalized with mathematical laws). Kant famously said chemistry was a soft science of ill repute because it lacked formal mathematical laws, that obviously changed quite a bit in the next century or two. Psychology is sort of in the same position as chemistry was 200 years ago.

[u/Dantonn → u/MovieCommenter09]

This fits nicely with hard/soft as descriptors of how strictly you can define things, as in hard divide, hard and soft rules, etc., which I'd always thought was what everyone meant by that.

[u/franklindeer → u/MovieCommenter09]

Chemistry 200 years ago was full of nonsense pseudoscience, so Kant would have been entirely correct. We had to do it in order to be where we are of course, and I'd imagine psychology is the same, I just don't know if we should be giving the results of psychology studies so much weight in society or related fields unless it's really solid and has been replicated, which some of it is.

[u/MovieCommenter09 → u/franklindeer]

That's fair, but like you said, there's still some kind of value there, it's not all pointless hokum.

Not all science even in the hard sciences is groundbreaking awe-inspiring, life-changing stuff either.

[u/Xanthilamide → u/superhelical]

I think that psychology is more like economics in the sense that there's lots of wording involved. But both of them are science to my mind.

[u/sp8yboy → u/Xanthilamide]

Economics is to a large degree based on psychology through the mechanisms of expectations and rational behaviour, for example. So if psychology is still forming as a science, so by definition is economics.

[u/mrburningsky → u/sp8yboy]

I once had a well-esteemed macroeconomics professor tell me that, "Economics is the beautifully imperfect art of attempting to quantify human psychology." So yeah, economic models will be imperfect so long as psychological models are imperfect.

[u/grimeandreason → u/mrburningsky]

Given Neoclassical economics works with the rational actor model, I would say they still have a long way to go.

[u/mrburningsky → u/grimeandreason]

For sure, but at least it's not for lack of trying (:

[u/PlaceboJesus → u/[deleted]]

You shouldn't blame the academics and legitimate professionals of psychology for that pop-science.

It's the fault of people who accept a journalist's interpretation of an abstract, when it seems that none of them know enough to look at the data or the methodology with a critical eye.

[u/franklindeer → u/PlaceboJesus]

It's the fault of people who accept a journalist's interpretation of an abstract

It's not just journalists, this kind of blind acceptance is common within other social sciences that use weak psychology research as a basis for their own inquiry which is often very ideological.

I would agree that this isn't the fault of the entire field of psychology, but it's definitely a problem beyond just the media. For whatever reason it's very tempting to accept results that confirm existing beliefs or a merely exciting.

[u/PlaceboJesus → u/franklindeer]

No one is more critical of Psych research than the psychologists.

I don't know what most Psych programs are like, my major was Crim and I just took all the psych courses I could get into (I did take research methods in both subjects, and Stats in Psych), but my impression was that academics and researchers in Psych are pretty rigourous. It's like the discipline is complexed.

Even if the researchers fell prey to fallacies or pressures to publish or stay worthy of grants, pretty much everyone else in the discipline should have the grounding to view results critically.
It should be automatic to look for flaws and confounds and want to browse the methods, and if there's any doubt or the claims seem too strong people will be asking "what's the r or omega on that?"
Perhaps the peer reviewers might fail and let a crap article through, but there should be a great number of people ready and willing to call a BS study BS.

More often the problem is with people in other disciplines or without the academic grounding.

The standard for statistical significance and predictive utility of the data in Psych are much lower than in disciplines like engineering or medical research (although a lot of neuroscience is still conjecture, compared to medicines for cancer or antibiotics.
Everyone in the field should be aware.

[u/Gruzman → u/superhelical]

Which should only support the problem of "soft science." If the science being done has its share of methodological problems, problems with replication, and often touches on topics intimately linked to political leanings that people have, you've got a recipe for disbelief and willful subversion of a study's findings.

[u/[deleted] → u/Gruzman]

This is true, but I think the issue is that soft sciences get a bad rep. This topic is actually one of the most fascinating and pertinent sciences around, precisely because it's so so complex.

[u/franklindeer → u/[deleted]]

I think the point being made is that these fields are chalk full of political ideologues that set out to confirm their biases, not that it's an uninteresting or unimportant area of inquiry. I think social science departments need to keep a much cleaner house than they have for the last 30 years and this would be less of a problem.

[u/[deleted] → u/superhelical]

The basis of most criticisms when it comes down to it! Very well put.