transcript of episode 23: 2021: A YEAR IN ERROR, 7th January 2022

[🎶 INTRO: "Spring Swing" by Dee Yan-Kee 🎶]

welcome to the error bar: where brain zingers are zapped

in this episode: how rocket scientists & brain surgeons are just as clever as you, how to publish anything in medical journals, how music affects the brain & 2021 has ended

here is the brain news on the 7th January 2022:



the British Medical Journal's Christmas issue always contains some 'fun' science, resulting in two stories at the error bar.

the first is that a survey & online intelligence test of 748 aeronautical engineers (rocket scientists) & neurosurgeons (brain surgeons) has found that members of these professions are in general no better at 12 cognitive tests that a large cohort of the general British population.

the researchers recruited rocketeers & neurosurgeons through professional mailing lists, inviting them to complete 12 computer tasks over 30 minutes. these tasks were taken from a battery of tasks completed by around 250 thousand members of the Great British Public in a previous study.

after data cleaning & processing, the brain surgeons were a bit better than the rocket scientists in one cognitive domain, & the rocketeers a bit better than the surgeons in another. compared to the population, surgeons were faster on one thing & slower at another.


are rocket scientists & neurosurgeons as smart as us?

yes. probably.

with the levity of presentation, the time of year & this story's popularity with the British Press, it's fair to say that this report provides no strong evidence that these two groups - of largely white European men - did not perform better than the general public on these tests.

yes, there were some 'significant differences' between the scores of the two groups of white male Europeans, & some differences between the neurosurgeons and the British, but these small differences should be taken into account along with the study's several flaws:

first, the two groups were quite different - in size & geographic origin; there were also many more men in the groups compared to the general population.

second, the data from 748 respondents was whittled-down to just 401, after processes of 'data cleaning'. this may be typical for online research studies, but it leaves the error bar with little confidence in the quality or representativeness of the cognitive test scores & reaction time data that remains.

third, the data were processed by running a factor-analysis, which reduces lots of different kinds of data into a smaller number of simpler scores. in this study, the factor analysis was done on the 401 new datasets, reducing data from 12 tasks down to 6 different scores. the scores of the general population were then converted & compared with the rocketeers & surgeons scores. this seems like the wrong way around - why not do the factor analysis on the more reliable & less-biased whole population data, then calculate the new participants' scores?

fourth, the two groups were compared with each other using 6 statistical tests, then each group was compared to the general population again with 6 tests. 18 tests were done, four were statistically significant, & only one would have survived corrections for multiple comparisons, if they'd been done.


the error bar is not lowered at christmas or any other time of year - this study provides only weak evidence that rocket scientists & neurosurgeons have similar cognitive skills to the general british population.


the science was by Usher et al. 2021: British Medical Journal; reported in The Independent by Vishwam Sankaran on 14/Dec/21, & The Daily Mail by @shivalibest on 13/Dec/21, & The Guardian by @NicolaKSDavis on 13/Dec/21



academics have a sense of humour, honest.

to prove this, several journals have traditions of publishing amusing or spoof articles at christmastime. a favourite of mine is Bones & Johnson's 2007 article in Perspectives on Psychological Science, explaining how psychologists are running out of naive participants for the Implicit Association Test, & arguing that tests on the unborn & the dead were now warranted.

in a similar vein, in the British Medical Journal, Greta Bauer is bemused by the volume of publications telling you how to publish, stretching to publications of guidance for drawing up publication guidance.

in response, Professor Bauer has produced BOGUS - the Biased Outcome reporting Guidelines for Underwhelming Studies (BOGUS) statement and checklist. you can use Bauer's very helpful wine-stained flowchart to decide how, what & where to publish your work, regardless of its merit.

it seems very likely that this will be used, by some, without irony.


very helpful


the science was by Bauer 2021: British Medical Journal;

and the brain in brief...



after six days of quarantine, the last thing i needed was a BBC radio 4 quiz host telling me that listening to music specifically releases dopamine in the brain, that dopamine is the 'happy hormone', that that's all there is to say on the subject of music neuroscience & that the contestant who answered 'serotonin' was, in fact, wrong.

i will not patronise the good nerds of the error bar by wasting precious seconds explaining why neither the question, nor the answer, makes any sense at all.


merry christmas


reported in The BBC by @PaulGambaccini on 27/Dec/21



the deaths of four prominent neuroscientists & evolutionary scientists have been announced.

Professor Henry Plotkin, former head of department at University College London, worked on evolutionary concepts including the Darwin Machine and Universal Darwinism.

Professor Teuvo Kohonen, a Finnish computational neuroscientist who devised the concept of self-organising maps in the brain, then coded it in Matlab.

Professor Edward O Wilson, the American naturalist famous for working on ants, for sociobiology, affectionately known as 'Ant Man' and 'Darwin's natural heir'.

& Professor Richard Leakey, Kenyan paleontologist, ivory-burning conservationist, double-amputee, & part-time politician, famous for promoting Africa as the birthplace of humanity.


rest in peace


reported in Twitter by @sophiescott on 17/Dec/21, & Twitter by @aivoAALTO on 14/Dec/21, & The BBC on 3/Jan/22, & The BBC on 27/Dec/21



the error bar is one year old, cheers!

a lot of podcasts & outlets produce an 'end of year show' around this time - the three that i regularly listen to produced '2021 in review' shows on average 5 days before the end of 2021, that's significantly earlier than year end; the t-values with 2 degrees of freedom is 8.66, and p=.013. please do write in to tell me how you can't run t-tests with less than 30 gazillion datapoints.

here at the error bar we wait until the year is finished before reviewing it. also i had covid over christmas, so really did not feel like dissecting neurobollocks between frequent throat clearances.

[for the rest of this story, please see here]


stay tuned for more error


2021: a year in error

well 2021 has gone, so what did we learn?

first, i learnt a lot. about processing audio signals (though i still have much to improve), about the colour of stars, rubber hands, journals, tools & the brain. for all this learning i thank my guests. i have tried to invite relatively early-career scientists to be guests on these shows, because these guys do most of the science & because we all know what the later-career people think anyway. if you know someone who could make a good story on or for the error bar, then let me know.

second, i have learnt a lot about what brain science gets published in the news media. it has nothing to do with the quality of the science, that is clear: there were some amazing - & some terrible - studies that made it into the brain science news. it seems to help if your science is freely-available: there were a disproportionate number of stories from open access sources like Frontiers, PLoS, the Royal Society & individually open-access papers. this seems obvious - why should the news media pay for science?

i have performed a principal components analysis on all the neuroscience stories published in 2021. the result was four components with eigenvalues over 1 that give all you error bar nerds some robust guidance about what you should be researching in 2022. principal component 1 is dogs: make sure your study is about dogs. component 2 is dementia: ideally dogs with dementia, or dogs who help people with dementia. the third component is sex & body odours: anything disgusting will do. fourth is tech: robots, AI, whatever nonsense mr musk said this week. conclusion: if you want to be in the media in 2022, it's time to get your robot dog to sniff out dementia from bodily excretions.

an end-of-year show would not be complete without prizes. & here are the golden bar of error winners for 2021:


my favourite story of 2021 was Havana syndrome. it has everything - spies, mind-rays, presidents of world superpowers making statements of denial & resolve, plus super-dodgy cold-war era pseudoscience done by old men in white coats who haven't seen the sun in years. more of that in 2022 please!

my favourite paper was 'Dump the dimorphism', by Professor Eliot & colleagues. i'm no expert in the topic & i know there are some arguments about it, but this paper is a Behemoth - over six hundred studies systematically reviewed & analysed, 150 manuscript pages. it takes an hour just to download the paper. i suggest you take a look!

my favourite error of 2021 was committed by the authors of the 'Inception is a reality' paper - neuroscientists who claimed that *they* can infiltrate people's dreams, who then wrote a follow-up paper complaining that major companies are doing exactly that, then naming, promoting & linking to that company's products over & over again in a highly-publicised open letter. it was either ironic, deliberate, or impressively short-cited. well done.

finally, the neural correlate of 2021 is... frontopolar cortex, for its services to the enjoyment that some people get from watching other people pop pimples. outstanding work, well done frontopolar cortex.

that was the year that was. in 2022, the error bar will continue doing much the same thing, with some minor tweaks.

making the podcast takes a lot of time - about 8 hours per episode - & i need a bit more time to write my own papers for the daily mail to review. so there will be fewer episodes in 2022 - probably 20 - & they will come out mostly during the 30 weeks of academic term. there will still be discussion episodes, but fewer - these take 2 or 3 days each to produce.

& finally: a year ago i remember being nervous about uploading my first episode. i felt a lot like i did when making my first website about 20 years ago - why would anyone want to listen to this? i remember wondering what it would take for me to give it up & what to continue. well, dear error bar nerds, i am pleased to say that the 10 listener threshold i set in January has been maintained throughout. the first year has had three thousand down-listens, about 100 per episode. we're big in Germany & Austria - danke schön - & most popular with 18 to 35 year olds - awesome.

the error bar is now closed for refurbishment & will be open again in February 2022. Happy Christmas!

[🎶 OUTRO: "Cosmopolitan - Margarita - Bellini" by Dee Yan-Kee 🎶]

it's closing time at the error bar, but do drop in next time for more brain news, fact-checking & neuro-opinions. take care.

the error bar was devised & produced by Dr Nick Holmes from the University of Nottingham. the music by Dee Yan-Kee is available from the free music archive. find us at the error bar dot com, on twitter at bar error, or email talk at the error bar dot com