transcript of episode 34: BIAS, FATIGUE, FRAUD? TAKE A HIKE, 11th December 2022
[🎶 INTRO: "Spring Swing" by Dee Yan-Kee 🎶]
welcome to the error bar: pushing neuroscience all the way to penalties
in this episode: how learning to play the piano improves perception, why the brain needs a break, image manipulation & the wonder of winter walking
here is the brain news on the 11th December 2022:
[🎶 BIG BEN BONGS 🎶]
PIANO LEARNING IMPROVES PERCEPTION
last week, The Independent newspaper reported on a study that learning to play the piano improves our ability to perceive sights & sounds.
the study from researchers at the University of Bath in the UK involved 42 people with no musical training being assigned to three groups for an 11-week training period.
one group had one hour of basic piano instruction every other week. numbers were placed on the keys to assist the learners. another group listened to music for each of the hours; a final group spent their hours reading without music.
after the training, the novice piano players were better at deciding whether the video and audio signals from a movie were synchronised or not. the authors concluded that their "study presents important evidence on the causal link between music training & audio-visual temporal processing"
[🎶 INTERLUDE 🎶]
does piano training improve audiovisual perception?
no.
before i begin my devastating critique of this under-powered fishing expedition with selective data analysis, over-exclusion of data & participant response bias, let me just say that the study was interrupted by the COVID-19 pandemic & i believe that all studies should be published in some form so that we can at least assess the evidence.
that said...
first, as the authors acknowledged, this was a small study. that's fair enough - perhaps the effects of learning to play a piano are so strong that only 14 volunteers are required in each group to find evidence for this effect statistically. the error bar does not practice N-shaming.
so while i'm not concerned that the study had only 14 people in each group at the start of the study; i'm a bit concerned that the published dataset was reduced from 42 to 31 people, due to the usual mix of people not completing the study &, of course, COVID. most concerning, though, is that from those 31 participants, a further 5 were excluded as 'outliers', leaving only 8, 9, & 9 in the final three groups. the error bar does practice outlier-shaming. the "important evidence" claimed by the authors of this study therefore comes from just eight of the original 42 people.
second, the researchers collected lots of different measurements of personality & task performance, but did not do any of the necessary 'corrections for multiple comparisons'. that upsets the error bar. was this really a study that focussed on audio-visual time perception? if so, why do so many measurements & tests to start with?
third, although the study lasted for 11 weeks, & data were collected every 2 weeks, only the first & last tests for each person were included. 5 of the 7 tests for each person were thrown away, presumably because there was no effect of piano training in this more powerful dataset.
fourth, for the psychophysics nerds listening, the question the researchers asked their volunteers: "did you perceive that the audio & visual signals were synchronised or not?" is not one that has a correct answer. since there was no objective way to know if the volunteers perceived the signals as synchronous or not, the answers given to this question are affected by what we scientists call 'response bias'. one person might have a high criterion or level of evidence required, to say that something is 'simultaneous' rather than out of synch. another person might have a low criterion. but the methods used in this study cannot disentangle whether people actually perceived the video as unsynchronised, or instead were just biased one way or another for whatever reason.
overall, the result from these 8 people could be that perhaps piano playing changes response biases? why that would be is a bit of a mystery.
conclusion
false positive
sources
the science was by Che et al. 2022: Scientific Reports; reported in The Independent by @clairehayhurst on 2/Dec/22
[🎶 BIG BEN BONGS 🎶]
YOUR BRAIN NEEDS A BREAK
winning the prize for most the obvious, pointless & late brain news story of December 2022, the Daily Mail tells us that "New research finds the stress you feel during back-to-back meetings is all in your head."
it's not new - the story is from April 2021. & where else would stress be, but in your head?
anyway, let's look at this press release from researchers at Microsoft published in April 2021. under the headline "Research Proves Your Brain Needs Breaks", Microsoft claims to have amassed one of the world's largest bodies of research evidence on the subject of how we work - collected during the pandemic.
& what is that evidence, i hear some of you ask? 14 people had a video meeting while the electrical signals from their brains were monitored using electroencephalography. on one day, the 14 people had four hours of back-to-back meetings; on another, meetings were broken up with ten minute burst of meditation, using the Headspace app.
in the results section - of the press-release - we are first told that 'beta waves' are associated with stress & that during continuous hours of meetings, beta waves increased. during ten minute breaks between meetings, they decreased. we are shown colourful maps of brain activity in the two conditions, where the power of electrical beta activity is represented on a scale - with no numbers - where the units go from 'less stress' to 'more stress'. stress was not measured.
second, we are informed that, when you take breaks 'frontal alpha asymmetry' is positive, but it's negative without breaks. positive frontal alpha asymmetry is associated with higher engagement. engagement was not measured.
third, we are told that beta wave activity increases at the beginning & end of video meetings & that this is bad because beta activity is stress. neither of the two graphs shown in the press release have obvious increases at the starts or ends of meetings.
to conclude, the press release quotes the director of the research group, Michael Bohan: "What makes this study so powerful & relatable is that we're effectively visualizing for people what they experience phenomenologically inside... It's not an abstraction — quite the opposite. It's a scientific expression of the stress & fatigue people feel during back-to-backs."
[🎶 INTERLUDE 🎶]
is this a scientific expression of stress & fatigue?
no.
Microsoft did not measure stress. or fatigue. or engagement. or anything, in fact, apart from electrical potentials on the scalp while people endured either four hours of video calls back-to-back, or took ten minute meditation breaks between calls.
did people close their eyes when meditating during these breaks? we don't know.
did people relax their head & neck muscles during these breaks, or just continue hunching & staring at the screen? we don't know.
what do the colourful images presented - by one of the world's largest & richest tech companies - mean, & what do they tell us? we don't know.
this press release, far from telling us anything useful at all, has removed knowledge from my brain. what's the point of having the largest body of evidence on a topic, if all that evidence is crap?
what i can work out is that, according to the images, during video meetings there was, on average, more electrical activity in the 'beta-band frequency' in the area of the head just behind & below the ears, as compared to during meditation. the beta band comprises oscillations of electrical signals with frequencies between around 13 & 30 times a second. probably - Microsoft didn't say.
muscle activity & movement is often related to electrical activity in the beta band, in things given names like beta-desynchronisation, beta-rebound, & beta-coherence. there are large neck muscles just behind & below the ears. perhaps people relax their necks when not staring at a screen for four hours straight? it's just a thought.
conclusion
too bad to be false.
sources
reported in The Daily Mail by Stacy Liberatore on 7/Dec/22
and the brain in brief...
[🎶 BIG BEN BONGS 🎶]
IMAGE MANIPULATION INVESTIGATION
the Scientist dot com reports on the ongoing investigation into whether the President of Stanford University's scientific work is affected by image manipulation, a kind of scientific fraud.
science is supposed to be about impartial evidence. sometimes scientists make mistakes & the scientific processes of pre- & post-publication peer review - like the error bar - are supposed to help find those mistakes & correct them. other times, scientists deliberately manipulate their data, or fail to correct mistakes. mistakes are normal & human; fraud in science is alien.
before Dr Elisabeth Bik started work, many biological images had been published with mistakes. & these mistakes often went un-corrected. Bik has built a career based upon checking tens of thousands of published biological images for potential errors, & reporting them when errors are found. this work has won Dr Bik many friends & many enemies. the error bar is a friend to Dr Bik.
in the latest controversy related to Dr Bik's work, the President of Stanford University, Marc Tessier-Lavigne is being investigated for image manipulation in four or more published papers.
if you'd like some fun activity to pass the time this winter, why not visit pubpeer.com to watch the flurry of activity over Tessier-Lavigne's papers?
the error bar will say no more on this issue for now, except for the fact that Dr Bik is our hero. well done & keep up the excellent work.
conclusion
stay tuned
sources
reported in The Scientist by @ki_irving on 30/Nov/22
[🎶 BIG BEN BONGS 🎶]
WINTER WALKING WOMEN WORK WONDERS
as the long dark nights have brought colder temperatures to the Northern hemisphere & as the snow flops down onto my attic windows, the Daily Mail was quick to report that going for a walk in the snowy countryside can make you feel better about your self & your body.
the study was carried out in a wintry forest on the southern borderlands of Poland. 4 men and 87 women, as part of their pedagogy course, completed some questionnaires about their body image satisfaction, then went for a 40 minute walk in the woods, having been instructed to 'behave normally'.
after the walk, the questionnaires were repeated. the 87 female participants felt more satisfied with their bodies after the walk.
[🎶 INTERLUDE 🎶]
does walking in the polish forest improve how you feel?
yes.
conclusion
a control group would help
sources
the science was by Czepczor-Bernat et al. 2022: International Journal Of Environmental Research And Public Health; reported in The Daily Mail by Fiona Jackson on 29/Nov/22
[🎶 BIG BEN BONGS 🎶]
MERRY CHRISTMAS
the bar has been very busy in the last few months. not opening, serving or talking to customers, but instead finishing work out the back, clearing the office, deconstructing the laboratory.
for the next month at least, the error bar will have no local server to run on. that's because i'm moving from Nottingham to Birmingham. with a timetable depending on my new office, IT & computing situations, the error bar will return, in Spring 2023.
the first two years of business at the error bar have been stimulating, but we are yet to make a profit. or any money at all, to be honest.
the bar's science news reporting has forced me to engage with research papers & topics i never would have. i have read a lot more of the Daily Mail than anyone should. i think i know a lot more now about how scientific papers get attention in the media. my conclusion is a timeless one for academia: it's nothing to do with quality.
i hope by the time the next episode is released that the error bar will have received an entirely arbitrary five thousand downstreams overall. if you want to help make that happen, please consider giving the gift of a subscription to the error bar this Christmas. it costs you nothing, & 100% of the imaginary profits will go directly to help Dr Holmes become slightly less disappointed with most of the brain science covered by the English speaking media.
take care this winter if you are in the Northern Hemisphere. please fight the spread of disinformation wherever you live & in all its many forms. why not start by refusing to promote the pernicious propaganda from the Father of Fake News himself, Santa Claus.
conclusion
happy christmas
sources
[🎶 OUTRO: "Cosmopolitan - Margarita - Bellini" by Dee Yan-Kee 🎶]
it's closing time at the error bar, but do drop in next time for more brain news, fact-checking & neuro-opinions. take care.
the error bar was devised & produced by Dr Nick Holmes from the University of Nottingham. the music by Dee Yan-Kee is available from the free music archive. find us at the error bar dot com, on twitter at bar error, or email talk at the error bar dot com.ε