transcript of episode 15: RABBIT HOLES AND P-VALUES, 20th August 2021

[🎶 INTRO: "Spring Swing" by Dee Yan-Kee 🎶]

welcome to the error bar: where obfuscation is ostracised

in this episode: as silly season continues, newspapers & science websites re-hash stories told decades ago, making more of the same errors, about p, alcohol, rabbits & divorce

here is the brain news on the 20th August 2021:

[🎶 BIG BEN BONGS 🎶]

THE NULL RITUAL OR: P, P, P, PLAGIARISM

the error bar has returned to deal with an article about p-values in Science News.org.

p-values are statistical tools that researchers have used for more than a century to indicate the likelihood, or surprisingness, of experimental results, assuming that there was, in fact, no real result at all. it's part of what's often called 'null hypothesis significance testing,' used in many disciplines to help decide whether experimental effects are real.

the author of this latest article about p-values, Bruce Bower, claims that psychologists have been misled by p-values for more than 70 years; that we are trapped in what he calls the 'null ritual'.

psychologists are guilty, Bower argues, of continuously and mindlessly testing the null hypothesis, of not looking at or showing our data, not reporting averages, not developing theories & not repeating experiments.

the sooner we abandon this ritual, he argues, the sooner we can do proper science. you know, like psychology in the old days - bearded chaps such as Pavlov, Köhler & Piaget.

[🎶 INTERLUDE 🎶]

is psychology stuck in the null ritual?

there is so much wrong & not to like about this article that i could happily fill a whole episode describing it. i'll spare you the monologue & just give a brief critique.

first, as a whole, this article is a naïve, stroppy, clichéd, hatchet-job of the entire discipline of psychology. it's not an argument but a string of unqualified, sweeping, often incorrect statements. forty years ago Psychology awarded Bower an MSc. Psychology should take it back.

second, this article has been published before. in 1997, Bower wrote a very similar article.

third, this article has been published before. in 2013, Bower wrote a very similar article.

fourth, this article has been published before. in 2018, Bower wrote a very similar article.

fifth, that's right - Bower has written similar articles, using the same sources & quotations four times over 24 years. his go-to guys are Loftus, Gigerenzer & Rodin. why did he feel we needed to read this yet again? he's like an arsonist wandering around a police station with a dripping can of petrol.

sixth, his errors. what a list! i'll pick the worst:

Bower claims that psychology textbook writers created the idea of statistical significance in the mid-twentieth century. they didn't. as his colleague Tom Siegfried wrote - better - in 2010, it began in the nineteenth century.

Bower states that "Statistically significant results are rarely used as a launching pad for testing alternative explanations". no Mr Bower, we psychologists: a) do more than one experiment, b) test theories multiple times over our careers & c) have a discussion section in every article.

Bower talks of a golden age of Pavlov, Köhler & Piaget - three high-school textbook psychologists - who, he claims, didn't use the concept of 'statistical significance'. i'm not going to fact-check that claim, because i'm certain that Bower has read all of their collected works in their original Russian, German & French.

one thing i did do, however, is read what Jean Piaget said about psychologists in 1979:

"the psychologist seeks to be as logical as possible in his reasoning and he borrows formulas from the theory of probability when he does his statistics"

conclusion

mr bower, it's time to stop your damaging ritual of publishing the same article again & again & again. psychology is a solid science doing wonderful things & we understand statistics a lot better than you.

sources

the science was by Piaget 1979: Annual Review of Psychology; reported in Science News by @Bruce_Bower on 12/Aug/21, & Twitter by @statsepi on 16/Aug/21

[🎶 BIG BEN BONGS 🎶]

A LITTLE ALCOHOL IS GOOD FOR YOUR HEART

in one of the only Daily Mail headlines without a single word in BLOCK CAPITALS, we learn that six pints of beer a week is good for our heart.

it's not a neuroscience article - apologies - but it is an article about beer, something the error bar is very interested in.

the claim is from a meta-analytic study of different levels of alcohol intake & cardiovascular health. across 12 studies & three databases, researchers pooled nearly 50 thousand people's data.

the conclusions seem well-supported: all levels of alcohol intake - up to a bottle of wine a day - seem to be better for you than not drinking at all. that's good news for error bars everywhere.

the only snag is that the 'not drinking at all' group included people who gave up drinking because of poor health. taking these people better into account will likely decrease the apparent health benefits of booze.

conclusion

cheers! [🎶 beer 🎶]

sources

the science was by Ding et al. 2021: BMC Medicine; reported in The Daily Mail by @jwillchad on 27/Jul/21

[🎶 BIG BEN BONGS 🎶]

DOWN THE RABBIT HOLE!

sitting on a grassy bank during a short break from my summer holiday at the University of Nottingham, one of our many resident rabbits approached me. it took out a pocket watch, looked at the numbers, looked back at me, twitched & hopped towards the Portland Building. i knew that i must follow it.

i followed the rabbit into the news agent & was led towards the Daily Express & a headline reporting that animals can recognize numbers just as well as children.

"Well! What a curious claim! I bet there's a solid piece of science behind this unlikely headline!" I found myself saying out loud like a pompous little girlie. lucky there's no-one else on campus.

in the second paragraph of this unlikely article, among other details, The Express reported that the report was first reported in The Mirror. "oh dear! I don't normally read The Mirror", i said, as a security guard approached.

so i followed the link & picked up The Mirror where, this time in the third paragraph, among other details about the very same story, The Mirror reported that the report was first reported in the Metro. "The Metro? but i don't normally read The Metro!" the worried security guard heard me say. "oh dear!", i said. "oh dear!"

so i followed the link & picked up The Metro where, this time in the thirty-seventh paragraph, The Metro reported that this whole report that the Metro had reported verbatim & The Mirror had reported was from The Metro & The Express had reported was from The Mirror, was originally reported in The Conversation.

and it was by Dr Silke Göbel, a number psychologist from the University of York. "oh dear! oh dear! don't journalists do any research any more?" I asked, as the security guard applied the cable ties to my wrists.

conclusion

follow the bunny

sources

reported in The Daily Express on 19/Aug/21, & The Mirror by @emilyreach1 on 9/Aug/21, & The Metro by Silke Göbel on 30/Jul/21

and the brain in brief...

[🎶 BIG BEN BONGS 🎶]

MATHS PREDICTS DIVORCE

in the second science news rabbit-hole this episode, i was drawn in not by an imaginary rabbit, but by the Express - again - telling us that scientists have now revealed an equation that predicts whether married couples will divorce. with 90% accuracy.

at first i was amazed - can this be true!? then i remembered that it was The Express - a tabloid misinformation outlet that is yet to post a single reliable story after 15 episodes.

as i fell down this second rabbit hole & back into the past, years & years went by & nothing happened. until 2015, when mathematician Dr Hannah Fry recorded a youtube video about the maths of marriage. then in 2014 she gave a TED talk on the maths of love.

i kept falling & falling until a 2004 BBC story & a 2003 book called the Mathematics of Marriage, where the 90% claim was made.

the original research papers where this equation was devised come from about 1995 to 2002. the abstracts say they used multivariate statistical methods to predict, with 80-95% accuracy, which of about 100 to 200 couples per study will divorce. this based on just a few minutes' interview with each couple.

[🎶 INTERLUDE 🎶]

can divorce be predicted with 90% accuracy?

i don't know.

the error bar does not fact-check 25-year-old psychotherapy studies. the only reason these 'stories' appeared in the newspapers is because they are trying to sell some pop-science maths brain-training gizmo. yes listeners, this is just an advert. after 15 episodes of fake science news, the Express & Mirror are permanently barred.

what i will do, however, is name the women involved in these maths studies. every single newspaper article, the TED talk & the youtube video named only the two men involved - let's call them Man A & Man B, but failed to mention the multiple female co-authors. only the BBC, in 2004, named one of them.

so, to Dr Catherine Swanson, Dr Kristin Swanson, Dr Sybil Carrère, and Dr Rebecca Tyson, may your marriages & your maths be bless-ed.

conclusion

spaff

sources

reported in The Daily Express by @LRStanfield on 3/Aug/21, & The Mirror by @emilyreach1 on 3/Aug/21

[🎶 OUTRO: "Cosmopolitan - Margarita - Bellini"by Dee Yan-Kee 🎶]

it's closing time at the error bar, but do drop in next time for more brain news, fact-checking & neuro-opinions. take care.

the error bar was devised & produced by Dr Nick Holmes from University of Birmingham's School of Sport, Exercise and Rehabilitation Sciences. the music by Dee Yan-Kee is available from the free music archive. find us at the error bar dot com, on twitter at bar error, or email talk at the error bar dot com.ε