Eyewitness: A lecture for charities on bad statistics

Stephen Cook heard the economist Tim Harford talk to a packed house about the pitfalls of statistical bulls**t and his desire to expand the "nerdsphere"

Tim Harford
Tim Harford

Another rebuke for charities, albeit relatively mild and humorous, came this week from the economist Tim Harford, who writes for the Financial Times and presents the BBC Radio 4 programme More or Less. His theme in a lecture organised by the charity Pro Bono Economics at the Royal Institution on Wednesday was "Good Causes and Bad Statistics", and he was clearly touching a nerve – more than 250 people, many from charities, crowded into the famous Faraday Theatre to listen to him.

Harford’s overture was about  bulls**t, hereinafter referred to as BS, which he defined as something different from lies. BS, he said, may or may not be true, but its truth or otherwise was not the point. "You find statistical BS all over the place, but nobody knows or cares what it means," he said. "And in the end, BS is a greater enemy of truth than lies are."

This took him to a story in the London newspaper the Evening Standard about how 25 per cent of guide dog users had been crashed into by cyclists. It made for a headline for Guide Dogs, said Harford – but after 24 hours it emerged that the figure was based on a "survey" conducted by putting out a tweet asking "are you visually impaired with strong views about cyclists?" Harford contended that the charity had acted naively and some information on sampling bias would have been appropriate.

His next example was a news story based on figures from the Stroke Association purporting to show that the number of people of working age suffering strokes  was "rocketing". The evidence was the rise in the number of people going to hospital in working hours with symptoms of stroke. But then it transpired that this was the result of a deliberate drive by the medical profession to get people to go to hospital immediately at the first sign of a possible stroke – and that the initiative was actually reducing the incidence of strokes. "Charities can get caught up in an idea and want to make a headline," said Harford. "Sometimes they’re just not very interested in whether what they are saying is true or not. Yet surely the truth should be enough to convince people."

His third example was the widely publicised Oxfam figure about how the 85 richest people have as much wealth as the poorest half of the world’s population: "The number is true, but it’s also BS because it doesn’t help us understand the big issue – nor was it designed to do that. The purpose was to drive people to the website and harvest their emails." He then entertainingly kicked around the way the figure was variously distorted and misreported, citing a graphic in The Independent that suggested that 85 people equalled 1 per cent of the world’s population. "If you see a figure and the first thing you have is feelings – anger, guilt, et cetera – it’s probably BS," Harford warned. "Try to have thoughts as well. Ask questions such as, is 85 people really 1 per cent of the world’s population? If you see a number, ask how they measured it – and if there’s no real answer, it’s probably BS."

In the panel discussion that followed, Dawn Austwick, chief executive of the Big Lottery Fund, accepted that charities sometimes made what were logical choices to use what she called "hysterical data" for purposes such as fundraising and PR. But she also knew of many charities, often small, that used "humble data" very effectively – "but it’s not very interesting for headlines".

Other contributors agreed that the contemporary world of instant, superficial information – soundbites, tweets and grabby one-liners – played a part in luring charities, along with other organisations, into the BS space.

Harford was asked finally whether this really mattered. Yes, he answered: "The trouble with BS is that it breeds cynicism and pollutes public discourse. And if we reward people for bringing out fun-sounding but misleading numbers, we will just get more of the same. So my aim is to expand the nerdsphere and get newspapers and others to ask more probing questions about statistics."

Before commenting please read our rules for commenting on articles.

If you see a comment you find offensive, you can flag it as inappropriate. In the top right-hand corner of an individual comment, you will see 'flag as inappropriate'. Clicking this prompts us to review the comment. For further information see our rules for commenting on articles.

comments powered by Disqus