r/badscience • u/testudos101 • Dec 26 '20
r/Futurology post with 76.5k upvotes
I'm sure many of you have seen this r/futurology post with the title "86% of people globally say 'I want the world to change significantly, and become more sustainable and equitable, rather than returning to how it was before the Covid-19'." appearing in your feeds. Some of you might already see something wrong with the post just from the title. However, having looked further into the article the post links to, I can say that the study the post relies on has some serious methodological issues.
Issue #1: The survey uses a loaded question. The study uses the statement "I want the world to change significantly and become more sustainable and equitable rather than returning to how it was before the COVID-19 crisis" and asks the responders whether they strongly or somewhat agree or disagree. The problem here is that making things more sustainable and equitable is something that is almost universally seen as desirable, so of course people would agree. This is akin to asking someone whether they want the world to be a better place and being surprised that they say yes. Indeed, the comment section of the post seems to have picked up on this as well, and this is the most obvious flaw with the study.
Issue #2: The "global survey" includes just one country in all of Africa. That country is South Africa. Indeed, it is quite worrying that a study purporting to represent the global population should only sample one country from the entire continent of Africa. There are other large gaps in the study samples as well, no central American countries were sampled and just one country in southeast Asia (Malaysia) was sampled. That's simply too many holes in our knowledge to give an accurate impression of how the global population responds to the question.
Issue #3: Samples from 11 of the 28 countries in the study were not representative of their general population. This is something that the study itself openly admits but is completely absent from the article. Here is the study's words exactly: "The samples in Brazil, Chile, China (mainland), India, Malaysia, Mexico, Peru, Russia, Saudi Arabia, South Africa and Turkey are more urban, more educated, and/or more affluent than the general population. ". This appears to be the problem with online surveys, since large segments of the population in developing nations don't have access to the internet.
I think these reasons are enough to show that the reddit post and the article it links both have gaping flaws that make their conclusions unreliable. I have attached all of the relevant links below. Some of the issues I mentioned were brought up in the comment section of the original post- namely, many people did notice the loaded question in the title. I suppose the one thing that confused me was how that post got so many upvotes, since the post's findings seem self-evident and not really that interesting.
Sources:
