a study of students who choose to study maths finds weak relationships with one of eight brain chemicals measured, having excluded lots of different kinds of data in many different ways. the effects in younger students were not replicated  

original article: Zacharopoulos et al., 2021 (Proceedings of the National Academy of Sciences of the United States of America), reported in: Irish News on 9th June 2021 & The Daily Mail by Sam Tonkin on 7th June 2021 & The Guardian by Sally Weale on 7th June 2021

this story was in episode 12 #brain #maths #education #QRP

the error bar says

the Irish News, Guardian & Daily Mail all report a study from the University of Oxford into the brain chemistry of children who study maths versus those who don't

the paper reports that the brains of 17-year-olds studying maths at high school level contain different amounts of a particular brain chemical than students not studying maths

these differences were not found in a group of children 2.5 years younger. the authors of the study claim that this particular brain chemistry develops significantly only in those who continue to study maths at high school

is brain chemistry development due to maths?

oh, dear listener, where do i begin?

i started reading this paper with suspicion & as i read, my bias was confirmed

the claim is that 14-year-old students' brains are different from 17-year-olds & that this difference can be explained by 2 or 3 more years of maths

if true, this would be important. so how good is the evidence?

first, while the data look impressive on the page, the largest statistical effects are all behavioural & educational - children who are good at maths are less anxious about it, do better at it & study it longer. the statistical effects in the brain data are all small. every single one of them is small - why?

second, in the 17-year-old group, there are 9 more maths than non-maths students, while the younger group is balanced & smaller - did data collection continue unequally in older students - why?

third, the authors removed so-called 'outlier' data in many different ways. they don't say how much data was removed overall, but Appendix 6 shows different amounts of data were removed on each of seven different criteria for each group separately - why?

fourth, the authors say they measured 8 brain chemicals, with 2 different baselines & adjustments for different brain structures. but only 2 chemicals are reported for one brain area & only 1 chemical for the other - why?

fifth, both groups were tested on three kinds of brain scan, but data from one of these scans was presented only for the older group - why?

sixth, statistical results that are 'not significant' & very weak 'trends' are nevertheless interpreted - why?

finally, the authors' claim depends on the younger, smaller group not showing the same statistical effects as the older group. an alternative interpretation is that this was just a failure to replicate the effects in a second group that was much too small, given the very weak effects found in the older group

the only positive here is that the data are available online if anyone wants to analyse them properly


weak & opaque


Irish News: fudge - scientific story distorted

The Daily Mail: fair - scientific story mostly intact

The Guardian: fair - scientific story mostly intact