Why “Brain Training” with Luminosity doesn’t work | The Stochastic Man

Why “Brain Training” doesn’t work | The Stochastic Man

Posted by thestochasticman under Bad Science | Tags: Bad Science, brain training, brains, Guardian, Lumosity, neuroscience, PR guff, Pseudoscience |
[257] Comments 

Mmm...brains, anyone?

Opening up my Guardian today, I was pleased to read their report on the Which? Magazine survey focusing on “Brain Training” games. You know the sort; the user is given a simple repetitive task to do, such as adding up numbers, and over time it will ‘improve’ your brain. Except that it won’t really. Well, not much more then any other activity, such as doing a crossword. That won’t set you back £30 for one of their games as well.

The Which? report found, with the help of several neuroscientists, that there was no peer-reviewed evidence to back up any of the manufacturer’s claims that they will improve performance or stave off dementia and other mental illnesses. The same level of improvement could have easily been attained through playing other computer games such as Tetris.

Fair play to the Guardian for being the only major British newspaper to report this, especially as they have a page on their jobs website sponsored by Lumosity in which one can, erm, ‘Train their Brain’ in order to “Improve basic cognitive abilities including memory and processing speed.”

'Train your brain' on the Guardian's own website

Lumosity have been mentioned a lot in connection with this report, as it is their very own Michael Scanlon who offered a rebuttal to their findings. “We would never say Lumosity is proven to improve day-to-day living, but there is more and more evidence it does. We have actually conducted our own clinical trials to measure effectiveness of the product”, he said using his own mouth.

It’s interesting that he said “We would never say Lumosity is proven to improve day-to-day living”, considering that in the picture above it explicitly claims that their games can improve memory (note the meaningless graph on the right-hand side) and ‘processing speed’, whatever the hell that is. Their own website is full of PR guff on how their games ‘take care of your brain’ (and presumably dry your tears when you’re feeling down as well).

The news article also lists him as a ‘neuroscientist’, which is repeated ad nauseum in other reports on this story. Again this is interesting as this factoid isn’t listed on Lumosity’s own website, and the only references to his neuroscience qualifications was this page, which lists him as an alumnus of Stanford University. So it appears he has the credentials, but I find it a bit weird calling someone a neuroscientist when they haven’t recently performed or published any research. Especially research into their own wondergames.

That’s not entirely true. Dig deeper and you will find this small internal study which appears to back up the companies’ claims that their games will makes you smarter. Except that it’s flawed to hell and back (which is presumably why it’s unpublished in any decent scientific journal). They took around 20 people and split them into two groups; one that performed simple visual and memory tests without playing their games (the so-called ‘control group’) and another which did whilst playing Lumosity games in between. So what went wrong?

1) Few participants. They only looked at 23 people, with 14 placed in the trained group and nine placed in the control group (and one of them dropped out). This is a tiny number to make up a scientific study with. There will naturally be so much random noise that it would be hard to pinpoint down any changes between groups. It’s not exactly hard to hire people to play games for a study, so there’s really no excuse.

2) Lack of a decent control. The only ‘control’ group did not play anything between taking tests. So how can the investigators be sure that it is the games themselves causing any benefit, or just the fact that a group of people were performing a simple task to increase awareness? A third group in which volunteers did crosswords or played Tetris in between tests would have sorted this out. Of course, this might show that Lumosity games are not very good, which would make for a horrendous marketing campaign, wouldn’t it?

3) Cooked results. One of their claims is that ‘memory span’ was increased after playing their games. Was it though? Their graph was presented as follows:



The study made a big deal that in the trained group there is a ‘statistically significant’ increase in memory capacity after playing these games. That is, the increase was very unlikely to happen by random. What about the control group though? The scores attained there are, in essence, exactly the same as the trained group. So these games don’t make a bit of difference to memory performance, in this regard. In fact I wouldn’t be suprised that just getting used to the tests lead to a naturally increased score, just by becoming accustomed to it.

Sigh. I’m getting tired now or marketing guff and the pointless word ‘Lumosity’. If you really want to mildly stretch your brain, why not do a crossword? The Guardian have a massive archive for you to try. (I have to linked to them as I do feel mildly guilty with the earlier comments; being a long-haired liberal, it is still my newspaper of choice.)

Finally, for more brain myths debunked, why not watch online the University of Edinburgh’s Christmas lecture, given by the excellent Sergio Della Sala? OK so it may take ages to load and still relies on Quick Time as opposed to Flash, but it’s a great way to learn more about why these marketing ideas are actually more likely to turn your brain to goo.