transcript of episode 19: DARWINIAN OVULATION, 29th October 2021
welcome to the error bar: where cerebral sophistry is squashed
in this episode: how women are more creative when ovulating, how to treat autism, and the error bar receives a rebuttal
here is the brain news on the 29th October 2021:
[🎶 BIG BEN BONGS 🎶]
OVULATE TO CREATE
reporting on a piece of tabloid gold, the Daily Mail tells us that 'women are more creative during the fertile stage of their monthly cycles'. that's a direct quote from the headline & i apologise in advance for the large proportion of listeners likely offended by this story...
...the paper, by social scientists in Warsaw, reports a study of 'signalling theory' - the Darwinian idea that male peacocks attract females by showing off their feathers. later theorists have argued that human creativity is, & i quote: an "artistic ornamentation beyond the body [which] extends the natural sexual human adornments such as penises, beards, breasts & buttocks".
to find out if women use their creativity like they do their breasts, 1045 women aged 18-35 were recruited through Facebook. those who reported being pregnant, having infants, breastfeeding, or being on the contraceptive pill were removed.
the women reported when their last period was, so the researchers could estimate when they were most fertile. during an online test, the women were then given five minutes to ask as many creative questions as possible about an ambiguous image on screen.
the data from 751 respondents showed that their estimated fertility was associated with more flexible & original thinking, but not with more fluency of thought.
[🎶 INTERLUDE 🎶]
does ovulation make women creative?
i'm going to stick my neck out here & say: no.
it's not just that i'm inherently suspicious of Darwinian evolutionary psychologists asking women to tell them about their last period, rather it's the quality of the study.
first, it's an online, self-report study. the researchers have no way of knowing that the people completing the study are, in fact, women of the target age group.
second, assuming that the respondents are indeed ovulating young women completing the tasks honestly & accurately, the data are of mixed quality. of the three creativity ratings used - fluency, flexibility & originality - the researchers agreed with each other very well about what the fluency ratings meant, but much less well on the other two. yet it was those other two ratings that showed the strongest effects of estimated fertility. it raises suspicions when the worst data are used to make the strongest claims.
third, figure 1 in the report, showing the data itself, is, er, creative at best, but a travesty at worst - it's not clear what the horizontal & vertical axes of the graph show. probability of conception, for example, is on a scale of 0 to 13. probabilities don't go up to 13! & for some unknowable reason, all the data on the vertical axis have been artificially shifted apart. if regression equations & error bars had even been given, i might be able to say more.
fourth, even if the respondents were all ovulating women, and all the data well-analysed, maybe, as acknowledged by the authors, women are a just bit less creative when suffering period pain, rather than being more creative when ovulating.
no, just no.
[🎶 BIG BEN BONGS 🎶]
AUTISM IS TREATABLE
The Week, a magazine that I read every, er, week, reported on the 2nd October that infants who display signs of autism are 'far less likely to later be diagnosed as autistic' several years later if they receive ten sessions of social interaction therapy over five months.
the effect of this remarkable therapy was that, at age 3 years, 'only 6.7 percent' of the children receiving therapy were diagnosed with autism, while 'one in five' of the control group were.
[🎶 INTERLUDE 🎶]
does therapy prevent autism?
it might do, but the numbers of children involved in this study does not inspire confidence. autism refers to a range of developmental conditions that may affect around 1% of people.
when media stories report figures like '6.7 percent' of a population, we expect that there are at least 1000 people being studied - why else would they give us the percentage figures to the nearest tenth of a percent?
in fact, this '6.7 percent' is three children. & the 'one in five' figure is nine children. why doesn't The Week just say three and nine?
is three very different from nine? well that depends on how many children were treated. in the paper, published in JAMA Paediatrics, they started with 104 children aged 9 to 14 months who displayed potential signs of autism. the children were divided into two groups of 52, half were given 10 sessions of social interaction therapy plus the usual care; the other half received just the usual care.
assessments were carried out at 12, 18, 24 & 36 months of age. the authors' main measure was the severity of autism symptoms, but they also looked at other factors, including whether children were classified as autistic or not.
some children dropped out of the study, so the final sample was 45 children who had therapy & 44 [sic] who had not.
as a nerdy stats fan, three out of 45 & nine out of 44 look to me like a weak effect of treatment. in the paper, the statistical tests they used were all 'one-tailed' - that is, they were only looking for improvements in the treated group & were happy to ignore any negative effects of therapy. not a good idea where children are concerned. indeed, if you do the more justifiable 'two-tailed' tests, then there are no statistically significant improvements in the treated group.
i also ran my own statistical test, dear listener. the humble chi-squared test, which sounds like a form of group meditation, is a simple way to test whether there is an association between a particular group of people & a particular outcome. putting the numbers 42, 3, 35, & 9 into the test gave me the following result...
... not significant.
that's right - the 'far less likely' reported by The Week was, in fact, 'not less likely' when analysed over a cocktail at the error bar. looking across the paper as a whole, there were three assessment times, multiple outcome measures, no corrections for multiple comparisons & a parade of unusually weak 1-tailed p-values which error bar nerds will enjoy: .04, .04, .02, .02, .02, .02.
autism is a collection of conditions that neuroscience & psychology is, as yet, unable to explain or effectively intervene on. the importance of the topic makes clear that small, weak studies like this one should be ignored, for now.
and the brain in brief...
[🎶 BIG BEN BONGS 🎶]
BALLS: A REBUTTAL
back in June, i reviewed what i called the worst brain imaging study ever published. in the study, researchers put brain imaging devices on footballers' heads & tried to measure brain activation data in the football field. this, for information, is not possible.
in my review, i pointed out that 86% of the data was removed, in various different ways. i also pointed out the very large number of statistical tests that were done & the lack of overall clarity in the analytic approach. the brain responses in the paper look nothing like what we should expect.
i concluded that this work was balls: impossible, unpublishable, pseudoscience. so bad indeed that the authors, reviewers & editors should all have been sent off.
this week, i received a rebuttal by email & on pubpeer.com. the authors pointed out, first, that it was only 59 percent of data that were discarded (not 86 percent); second that, yes, there was no clear brain response in the data as a whole, but this is because all of the individual data was averaged together & the effects were smoothed out.
even under the most generous interpretation of this rebuttal, the authors discarded 59% of the data & did not find any consistent responses.
it's closing time at the error bar, but do drop in next time for more brain news, fact-checking & neuro-opinions. take care.
the error bar was devised & produced by Dr Nick Holmes from the University of Nottingham. the music by Dee Yan-Kee is available from the free music archive. find us at the error bar dot com, on twitter at bar error, or email talk at the error bar dot com.ε