Easter Statistics

Ah Easter; eggs, bunnies, desperately gloomy teachers' surveys and revelations about the Shroud of Turin. And how, apart from the season, are these last two areas linked, you may ask? Like this. If there's space we may even take in prostitution at the end.
Let me begin with the relic. This weekend the story emerged - midwifed into public view by L'Osservatore Romano, the Holy See's own newspaper - of how Barbara Frale, a researcher in the Secret Vatican Archives, had solved the mystery of where the linen supposedly bearing the image of the crucified Christ had been between the early Middle Ages and its “reappearance” in the 1350s. A series of documents concerning the 1309 trial of the Knights Templars (oh, do keep up) that she had found some time ago had now, in timely fashion, revealed how one interrogated knight had spoken of being shown a long cloth with an image of a man upon it.
From this it appeared safe for her and the Pope's newspaper to conclude that, after the sacking of Constantinople in 1204, when a shroud bearing the image of Christ was supposed to have disappeared, the Templars had had it in safe and secret keeping for a century.
The only problem is that there was no mystery to solve, because for 20 years we have known from carbon dating - or thought we have - that the shroud actually began life roughly when it “reappeared”. So there wasn't a missing century to account for.
Or was there? Last Easter BBC Two showed Shroud of Turin, in which an American physicist questioned the original carbon dating. The producer of the documentary, Performance Films, put out a press release headed “Top scientist calls for reinvestigation into authenticity of the Shroud of Turin” and quoted Professor Christopher Ramsey, director of the Oxford Radiocarbon Accelerator Unit - which carried out the 1989 carbon dating - as admitting that there was “a need to have a critical look at the evidence that [other experts] have come up with in order for us to try to work out some kind of a coherent story”.
Mystery reopened? No. Bizarrely, the less-noticed Oxford University press release on the same topic was entitled “International radiocarbon dating experts confirm the Turin Shroud is a medieval fake” and, though including Professor Ramsey's very proper willingness to conduct further research, added that “the researchers conclude the original radiocarbon date of 14th century is correct, based on current evidence, but they have yet to test whether there is anything in the specific storage conditions of the shroud which might affect this conclusion”.
Somehow Performance Films had also overlooked Professor Ramsey saying that there was no “direct evidence to suggest the original radiocarbon dates are not accurate”.
So there go two Easters' worth of shroud stories in one go, and now it must be time for the teachers. This is simpler and concerns the widely reported survey undertaken by the Association of Teachers and Lecturers on the subject of parental shortcomings and released to fanfare over the weekend.
“Research reveals that children and parents are becoming increasingly confrontational with teachers”; “teachers believe pupils' behaviour has worsened in the past few years, a survey has found”; “almost a quarter said they had seen pupils being physically violent, either towards the teacher or another student” and so on. Mary Bousted, the ATL general secretary, was reported as saying “it is shocking that over a third of teaching staff have experienced aggression from students' parents or guardians”.
Easter is when the several competing teaching unions take their crosses to the seaside, and there is usually a survey or two to publicise the various conferences. But when I looked at reports of the ATL survey, I couldn't see which pollster or market research company had carried it out. So I contacted the ATL's very co-operative press person. It turned out that this is how the ATL conducts surveys. First it somehow generates a completely random sample of 10,000 members, all of whom it e-mails with survey questions such as “has pupil behaviour got worse or better in the last five years?” Second, when it feels that there are enough answers in (1,000 in this case), it simply closes the poll.
I called one of the country's top pollsters to evaluate the ATL's method. He laughed. In the first place, he said, unless efforts were made to ensure that the original sample was representative (his organisation uses 16 variables) then the whole process “was not statistically valid”. This combined with closing the poll arbitrarily meant that the sample was merely of those “most motivated, active or randomly available”. His summary was “it is impossible to calculate a meaningful result out of it”.
Too late. The meaningless results featured prominently in the newspapers and got a nice juicy outing on the Today programme.
But I discover we do have room for the sex industry, so let's throw that in too. A month ago I asked readers for examples of bogus statistics and numbers that had somehow become viral. I have to admit that this issue of bad stats has become something of an obsession for me, not least because we seem to be having entire debates and advocating important policies on the basis of poor research. Anyway, one of my respondents was Hilary Kinnell, a well-respected author and researcher in the field of prostitution and sex work. Ms Kinnell pointed out that the Home Office routinely uses the figure of 80,000 for the estimated number of prostitutes in the UK. It did so again last November in its latest review document, Tackling the Demand for Prostitution.
She knew, she told me, where the 80,000 had come from, and how reliable it was. It had come from her and was completely unreliable! A decade ago, working in the impoverished voluntary sector, she had tried to make the calculation by asking about a third of the local organisations on her mailing list to estimate the number of prostitutes in their area. Under half replied. The average of those that did was 665 sex workers, so Ms Kinnell multiplied by 120, producing a figure of 79,800.
“This method was extremely crude,” wrote Ms Kinnell and was likely to be an overestimate. “Despite this,” she added, “the figure of 80,000 is constantly cited as a fact, without any qualification, often in the context of claims that the sex industry has expanded rapidly over the past ten years, often applied only to women and sometimes only to street sex workers.”
She's right. Google it and see. Honestly, you're better off believing in the bunny.