three groups of 14 people learned to play the piano, listened to music, or read quietly. the study was interrupted by covid. by the end, 8 people learning the piano got a little better at saying whether audio-visual signals were in synch.  

original article: Che et al., 2022 (Scientific Reports), reported in: The Independent by Claire Hayhurst on 2nd December 2022

this story was in episode 34 #responsebias #fishing #power #outliers

the error bar says

last week, The Independent newspaper reported on a study that learning to play the piano improves our ability to perceive sights & sounds.

the study from researchers at the University of Bath in the UK involved 42 people with no musical training being assigned to three groups for an 11-week training period.

one group had one hour of basic piano instruction every other week. numbers were placed on the keys to assist the learners. another group listened to music for each of the hours; a final group spent their hours reading without music.

after the training, the novice piano players were better at deciding whether the video and audio signals from a movie were synchronised or not. the authors concluded that their "study presents important evidence on the causal link between music training & audio-visual temporal processing"

does piano training improve audiovisual perception?


before i begin my devastating critique of this under-powered fishing expedition with selective data analysis, over-exclusion of data & participant response bias, let me just say that the study was interrupted by the COVID-19 pandemic & i believe that all studies should be published in some form so that we can at least assess the evidence.

that said...

first, as the authors acknowledged, this was a small study. that's fair enough - perhaps the effects of learning to play a piano are so strong that only 14 volunteers are required in each group to find evidence for this effect statistically. the error bar does not practice N-shaming.

so while i'm not concerned that the study had only 14 people in each group at the start of the study; i'm a bit concerned that the published dataset was reduced from 42 to 31 people, due to the usual mix of people not completing the study &, of course, COVID. most concerning, though, is that from those 31 participants, a further 5 were excluded as 'outliers', leaving only 8, 9, & 9 in the final three groups. the error bar does practice outlier-shaming. the "important evidence" claimed by the authors of this study therefore comes from just eight of the original 42 people.

second, the researchers collected lots of different measurements of personality & task performance, but did not do any of the necessary 'corrections for multiple comparisons'. that upsets the error bar. was this really a study that focussed on audio-visual time perception? if so, why do so many measurements & tests to start with?

third, although the study lasted for 11 weeks, & data were collected every 2 weeks, only the first & last tests for each person were included. 5 of the 7 tests for each person were thrown away, presumably because there was no effect of piano training in this more powerful dataset.

fourth, for the psychophysics nerds listening, the question the researchers asked their volunteers: "did you perceive that the audio & visual signals were synchronised or not?" is not one that has a correct answer. since there was no objective way to know if the volunteers perceived the signals as synchronous or not, the answers given to this question are affected by what we scientists call 'response bias'. one person might have a high criterion or level of evidence required, to say that something is 'simultaneous' rather than out of synch. another person might have a low criterion. but the methods used in this study cannot disentangle whether people actually perceived the video as unsynchronised, or instead were just biased one way or another for whatever reason.

overall, the result from these 8 people could be that perhaps piano playing changes response biases? why that would be is a bit of a mystery.


false positive


The Independent: fair - scientific story mostly intact