half an hour of weak electric brain stimulation for five days seems to decrease reports of obsessive and compulsive behaviours for three months, but is it too good to be true?  

original article: Grover et al., 2021 (Nature Medicine), reported in: The Daily Mail by Shivali Best on 18th January 2021 & Scientific American by Diana Kwon on 25th January 2021 image source

this story was in episode 2 #OCD #TACS #brainstim #brainhacking #reward

the error bar says

both Scientific American and The Daily Mail picked up on a remarkable report of an apparent treatment for obsessive-compulsive-disorder. while the Mail focused on the potential to cure problem gamblers, shopaholics, and compulsive eaters, Scientific American stuck a little closer to the actual story.

the study was in two parts. first, three groups of 20 healthy young people were given one of three kinds of electrical brain stimulation over the orbitofrontal cortex - an important part of the brain. two kinds of stimulation were expected to do nothing, and the third was expected to interfere with the volunteers' decision-making skills. their task was to choose one of two random characters on screen, learning which to choose by trial and error. sometimes they would be rewarded with $10, other times they would lose $10.

the brain stimulation worked amazingly well - the placebo and the lower-frequency stimulation didn't seem to affect people's decisions, but the higher-frequency stimulation really impaired their ability to learn from rewards.

second, 64 people were split into two groups of 32. they reported their obsessive-compulsive symptoms in a questionnaire at the start of the study, then had brain stimulation for 5 days, then repeated the questionnaire immediately afterwards, and again 1, 2, and 3 months later.

the results were equally impressive - very large reductions in obsessive-compulsive behaviours were reported for the higher-frequency stimulation group.

does brain stimulation really cure ocd?

despite the Mail's headline, this study involved no-one actually being treated for - let alone CURED of - OCD. and in the first half of the study, they also didn't measure any OCD behaviours. let's take the two halves of the study separately.

the first half reported data showing extremely large and strong effects of the brain stimulation - within the first few minutes of the experiment, effects are already clear and the error bars are tiny - the volunteers just can't seem to learn from rewards (receiving $10), whereas they can learn totally fine from punishments (losing $10). for the rewards, half of the group were performing individually at chance, and half better than chance.

if true, this result would be an enormous and very specific effect of brain stimulation on behaviour. and that's the only reason i have for not really believing it - it's just too strong to be true - such a massive effect, changing a group of people's decision-making behaviour so strongly and so specifically. it needs to be replicated.

the headline-grabbing part 2 of the study was about how the same kind of brain stimulation reduces obsessive behaviours. and just like the first part, the effect-sizes are very large, with participants consistently reporting reductions in obsessive behaviours over as long as 3 months.

but some things are gnawing at me.

first, the authors do a classic 'double dip' - by showing a strong relationship between the scores at baseline, and changes in those scores from baseline to afterwards. this is a topic for a later episode, but it's a common statistical mistake.

second, there are some minor errors in their calculations. they kindly provide all the raw data, but the data don't quite match up with their graphs. not a big problem, but a worry.

third, the group who received brain stimulation continued decreasing their obsessive behaviours months after treatment - there seemed to be no wearing-off. that just seems a bit odd.

fourth, the treatment group showed a very large increases in variability after treatment. why would obsessive variability increase - it could be that some people had indeed been 'cured' but some were unaffected or made worse.

fifth, colleagues have suggested that this form of stimulation could be quite uncomfortable, and would likely produce flashes of light in the eyes, although this was ruled out in the paper.

sixth, along with this increased variability for the treatment group, the relationships between the untreated participants' scores seemed a bit too good. perfect every time.

this skepticism may just be effect-size jealousy, but there's just something too good to be true in these data. two massive, perfect effects in one paper? hmmmm.


electrical brain stimulation seems to have massive and highly-specific effects on people learning about monetary rewards but not punishments, and on obsessive-compulsive behaviours. replication required


The Daily Mail: fudge - scientific story distorted

Scientific American: fact - scientific story reported well