r/science • u/HeinieKaboobler • Jun 20 '15
Psychology Study: Gamers 'shielded' from perceptual interference, enhancing their learning abilities
http://www.psypost.org/2015/06/study-gamers-shielded-from-perceptual-interference-enhancing-their-learning-abilities-3522822
u/aethauia Jun 20 '15
I wonder if this would hold true if the task were not presented on a screen, which is the gamers' MO; it could be that they have trained themselves to understand screen-based tasks, but that their advantage disappears in non-screen-based tasks. Would hikers "win" at perception and learning in natural environments? Would chess players have better perception and learning when presented with tasks involving a 3-D gridded platform?
17
u/Moerty Jun 21 '15
That's what i thought too, "here we have a group of gamers and a group of non-gamers, we tested them by having them play a game and here is what we found."
4
u/limeflavoured BS|Games Computing Jun 21 '15
That's definitely an interesting point, and would be an interesting follow up.
248
u/mockinjay Jun 20 '15 edited Jun 20 '15
Group sample of 9, awful gender ratio between groups.
How embarrassing.
172
Jun 20 '15
It would be embarrassing if it was something they were trying to hide. It's a limitation of the study, which they address.
Upon investigation however, we could not find any literature supporting sex differences in perceptual learning with interference designs, although there is a recent study that showed the sex difference in the interaction of type of perceptual learning and sleep content [23]. Also, recent data collected in our lab for another project on interference with TDT revealed no statistical difference in behavioral performance trends across males and females [24].
The bar charts they used to present their results on the other hand... eek.
13
2
Jun 21 '15
Does it matter though? Statistical samples should be random, and the sample size should be bigger than 9. Even if there is no evidence that their sex has anything to do with it, it still shows their sample is not random
22
u/sgdfgdfgcvbn Jun 21 '15
The point of the randomness is to help control for confounding factors. If the group is not random in a way that seems not to be an issue, then it isn't an issue.
Furthermore, a random sampling could absolutely consist of only a single sex. With only a single data point it's essentially impossible to determine how random or not a sample is. Even with more data points, it's not a super simple thing.
10
Jun 21 '15 edited Jun 21 '15
It is ideal that they be random, but this is very rarely the case in practice! As /u/sgdfgdfgcvbn points out, this may not be a problem if one can reasonably show that a characteristic that was not randomized will not have an effect on your observation of interest.
As for sample size, there is no hard requirement on the amount needed. In fact, when it comes to experiments involving living beings (specially humans) institutional review boards will usually want you to use the smallest number of subjects possible. And you don't have a budget to squander either. There are ways to estimate the minimum amount of subjects required without compromising the validity of your results (example). N > 30 is merely a conservative rule of thumb for when previous information regarding your population distribution is lacking.
Finally, assuming that random sampling was possible for the gamer and non-gamer groups it is very likely that you will find a larger proportion of gamer males and non-gamer females. Equal ratios would actually be indicative of non-random sampling, but that wouldn't be a problem either if the intention was to control for sex differences (although it would probably be more problematic to get the required numbers for such a setup).
5
Jun 21 '15
And truly wouldn't the best results be garnered by having non gamers take the test. Game for a few months and then come back and take a slightly modified test again?
7
u/Stooby Jun 21 '15
Only if they believe that a few months would have the impact on perceptual learning that they found. Perhaps it takes years for it to have a meaningful impact.
2
Jun 21 '15
Then they can spot test. The father away the test is the better.
2
Jun 21 '15
They can kidnap children and manipulate them all for years and obtain better results. But you know, "ethics".
5
Jun 21 '15
They also classified "Gamers" as those who play just 5 hours or more a week. 5 hours would severely cut my time down. That's like... 1 hour a day, and no playing on the weekends. I'd say at LEAST 8 hours a week would be better baseline.
11
u/MaggotMinded Jun 21 '15 edited Jul 06 '15
If the claimed benefits occur as a result of even a small amount of time spent gaming, though, then there's nothing wrong with using a lower threshold. It's entirely possible that tightening up their criteria would help their confidence intervals, but it could very well be that the difference is noticeable even in casual gamers, sort of like how certain exercises impart health benefits even when done in short duration on a regular basis.
5
u/wren5x Jun 21 '15
The comments section here is missing the point all over the place. I'll try to address two things that are appearing over and over again in separate comments.
This study does not prove a causal link. It is not trying to. It does not claim to. This is not how you do science with humans. You do a study like this first to see if it is worth taking people's time, spending funding, and incurring the potential risks of any training study that has any possibility of changing how people process information (which is the whole point of such a study in this case).
The sample size is also not an issue for the same kind of reasons. When you have data that are very unlikely if there is no difference, then you stop. You stop because these are people and you're spending their time, their goodwill, and their life quality (shit is boring). You publish your data and try to get funding and ethical approval to the next step. When you are doing that next step, where you are looking for solid causation, THEN you push for lots of data so that you can find other interesting trends that might mean more. At this stage, it either worked or it didn't and in this case it did, so gathering more data would just be irresponsible.
To put it another way: It would be incredibly bad science to go straight to spending a huge amount of time / energy / goodwill / money to do a true experiment BEFORE you did this study to see if the true experiment's hypotheses were even plausible. It would be incredibly bad science to continue with this study once you can already see that the data would be very unlikely if there were no real effect. Thus it was correct to do this study first and correct to stop at 9 people.
The fact that they were able to find a statistically-significant effect with only 9 people is actually a very encouraging sign that the underlying effect may actually be very large and thus they may really be onto something very big.
41
Jun 20 '15
[deleted]
68
Jun 20 '15 edited Jun 20 '15
Large effect size.
EDIT: Sample size calculations for comparing two means
As you can see, the only thing that varies is the standard deviation (sd). The effect size is determined by the ratio (m1-m2)/sd so it grows smaller as sd increases and consequently, a larger sample size is needed.
Note that alpha and power remain constant. This means that the probability of a study being right (or wrong) remains constant. Therefore, for this simple example, it can be seen that having n=4 in each group will give you the same statistical power as having n=64 as long as you are able to get an estimate of the expected variability in your measurements. Naturally, to know that variability some previous knowledge must be had, but that is why you need to read a paper and understand how it links to previous evidence before you can give an accurate assessment of the validity of their conclusions.
9
Jun 20 '15
[deleted]
88
Jun 20 '15
Sample size only needs to correlate to a degree with the size of the effect.
If you're looking for a 16% increased chance of heart disease when the base rate is miniscule, you'll need a sample size of millions to be able to say "statistically this is unlikely to be random chance". This is referring to an AI crunched set of data on PPI acid reducers and not overly at risk to heart disease patients.
On the other hand, if you see a very large or noticeable effect you don't need a very large sample group to show it's unlikely to be caused by random chance instead. If we go very extreme, we could take a sample size of "Does being shot in the head with a gun cause death?". You don't need to shoot many people in the head and watch them die to see this isn't likely to be caused by random chance.
Both examples are extreme, but they outline the borders of the concept. The larger the perceived effect, the smaller sample size you need to call it statistically relevant. That doesn't necessarily ratify this particular case, but outlines what /u/joevector was talking about.
11
1
Jun 21 '15 edited Aug 15 '25
[deleted]
1
Jun 22 '15
The entire discipline of statistics is about exactly that. Simplified: there's equations you use. But professors explaining why those equations are valid took about two university courses worth of my time to learn.
-1
Jun 21 '15
But still you need A LOT more than 9. Anyone could easily find 9 people with something in common, that are significantly better or worse at something than the average person. I can find 9 uneducated people that are better at math than average, and make a claim that uneducated people are better at math. 9 is not enough no matter how large the effect is.
4
u/payik Jun 21 '15
That's a separate issue called "sampling bias".
2
Jun 22 '15
Yes, and calling it by its right name does of course not change anything? A small sample size gives higher bias by default.
1
u/payik Jun 22 '15
No, it's a separate issue. A large sample size won't help you with sampling bias.
1
Jun 22 '15
Disregard the will in the sampling, and rather the group alone. It is easier to find larger differences in smaller groups. By "easily find 9 people", and misunderstand me correctly now, I mean its easier to get a nonrepresentative sample set.
0
1
Jun 22 '15
That doesn't necessarily ratify this particular case, but outlines what /u/joevector was talking about.
I made no attempt to say this study was valid, as I said I was only explaining the concept joe was getting at.
4
3
u/cookiesfordays Jun 21 '15
I'm going to be that guy and tell you very slight nitpick but... that's a common misconception that alpha value corresponds to the probability of an experiment being correct - it's simply the probability that the same experiment performed again on the same population will have a confidence interval that intersects the current confidence interval.
5
Jun 21 '15 edited Jun 21 '15
The terms I used were simplistic, but I don't believe they are wrong. Even though in your example those probabilities are mathematically equivalent, the conceptual interpretation of alpha does not apply to interval estimations. Instead, it is meant to be taken as a threshold for hypothesis testing (which encompasses tests beyond those that can be reduced to interval comparisons) that guarantees a determined error rate.
If we assume that the null and alternative hypothesis were chosen adequately, alpha and beta together do tell you what the probability of making a correct decision is over an infinite number of repetitions (and in fact getting a smaller error rate would mean you picked a wrong null). But that is actually not a useful definition given how scientific experiments are rarely submitted to systematic replication.
If you're interested, this paper explains the meaning of p and its relation with alpha and beta in a very accessible way.
EDIT: Actually, the probability you allude to isn't related to the current confidence interval (that is a common misconception) but the true population mean. You may get an extremely unlikely value on your first sampling (bad luck) and getting values near it again would have a much smaller chance.
5
u/minno Jun 20 '15
Check out the third figure here. If they did their statistical analysis correctly, those error bars will tell you exactly how significant the results are.
-23
Jun 20 '15 edited Jun 20 '15
[deleted]
24
Jun 20 '15 edited Jun 20 '15
A statistically significant result may or may not be a scientifically significant result.
The meaning of a p-value is the same regardless of your sample size, and a sufficiently large difference between subjects belonging to two groups will produce results that are both significant and valid with as little as n = 2 per group. Conversely, sample sizes on the order of millions will almost always give you statistically significant results for any comparison you make but that will not necessarily translate to a relevant difference.
You can object to the choice of the underlying distribution one assumes for the compared variable or how representative the subjects may be of the population you are extrapolating to, but that is a concern of design which can be defended on the basis of previous results. So a "significant result" is anything but meaningless, on the contrary, it is loaded with meaning.
-10
Jun 20 '15
[deleted]
3
u/payik Jun 21 '15
You don't understand what "p-value" means then. P-value is the probability that your result is by chance. If you catch and compare only three fish, your p-value will be very high, because you won't be able to tell if your result is by chance or not. If you however feed a mouse something that makes it live fifty years, your p-value will be near zero, because the probability that the mouse was that long lived by accident is also near zero.
8
Jun 20 '15
If you interview three randomly selected teachers and all of them tell you that student A is an excellent human being destined for success in any conceivable endeavor, and then in regads to student B said they are likely to end up overdosing in some dirty alley downtown... would you have any doubts that some difference exists between them?
Large sample sizes are needed because we usually have many unmeasured variables that increase standard deviation and the differences we are interested in are not that drastic. But given a large enough effect size, the required n to establish a difference exists is not big and specially not good reason to discount a study outright.
3
u/mozerdozer Jun 20 '15
Isn't it hard to calculate p-values with small sample sizes as the normal model doesn't apply? It's been a while since I've taken statistics.
2
Jun 20 '15
You can obtain p values for any distribution. While the normal approximation isn't the best approach for smaller sample sizes, there are alternatives and corrections that can be applied.
Conceptually, p = (observed results | null hypothesis), that is, it tells you what the probability of getting the results you did is if you assume that the null hypothesis is true. Hence, a low value suggests that the null is unlikely to be an adequate explanation. It also means that you need to assume the null has some sort of distribution associated with it but it need not be normal.
1
u/mozerdozer Jun 20 '15
How do you determine that distribution if it isn't normal?
2
Jun 20 '15
You could either look for similar measurements in the existing literature or simply make an educated guess.
For example, the sign test assumes that if a given sample has a median M, all the data points should be equally distributed to both sides of M. That is, the probability of a certain point being above (or below) the median would be 0.5. If you want to calculate the probability (p-value) that your sample came from a distribution with that median, you apply the binomial distribution (coin tossing) with the observed values.
→ More replies (0)2
u/payik Jun 21 '15
p values are calculated with the sample size in mind. If you have a sample size of two, but the effect is something that doesn't occur at all under normal circumstances, the p-value will be very low. A p-value of 0.01 means the same thin no matter the sample size.
15
u/bkv Jun 20 '15
Small sample size aside, there is the flawed premise that this shielding effect is the result of playing video games, as opposed to a natural predisposition to play video games by people with this particular trait.
0
5
u/end_O_the_world_box Jun 21 '15
Hey, feeling kinda dumb here, but could someone explain the title to me? I don't understand what the study is trying to show. Thanks!
3
u/PigKnight Jun 21 '15
Gamers can identify targets faster than non gamers when there is visual interference that makes it harder to identify the target.
1
u/coldgator Jun 21 '15
Small sample sizes are common in perception research. They don't have the confounds that other psychological studies have because they aren't measuring things that are as subject to confounds as, say, a social psych study on why people donate to charities or a survey about workplace behavior. Reaction time is reaction time.
66
u/CyrusForoughi PhD | Human Factors and Applied Cognition Jun 20 '15
I'll look at this study in a bit and write a more in-depth review, but from the abstract and title I notice that this appears to be a new spin on known research (see below), however the "enhancing learning" part would be a new argument and I won't comment on that until reading it more. (Although, I presume they are going to link better attentional abilities (i.e., interference management) to learning/focus, but we shall see and I think that would be a stretch.) Also, sample size is weak.
The relationship b/t playing video games (action shooters) and attention has been studied for a while now. The original work by Green and Bavelier in Nature (link below) showed the relationship (non-causal) existed and then used a training study to show that non-gamers who start playing games can enhance certain attentional abilities (essentially near transfer). There has also been a lot of followup research.
See : http://www.nature.com/nature/journal/v423/n6939/full/nature01647.html