24
votes

enter image description here

Comedian Stephen Wright once joked that 42.7% of all statistics are made up on the spot.

Recently, Arizona Senator and notably stubborn Jon Kyl provided us with the most famous example of just such behavior.

He can be seen here, stating that well over 90% of what planned parenthood does is abortions, despite the fact that the real amount is 3%. Later, he attempted to claim that his words were "not intended to be a factual statement" once it became clear he had just made up the 90% figure.

enter image description here

In a perfect world, all claims would be appropriately and accurately referenced, but we all know this just simply isn't the case, especially when it comes to politicians, the media, and those pushing an agenda dishonestly.

Has there ever been any scientific examination into how often quoted statistics are just made up? Do we honestly have any way to figure this out?

I assume any study of this would have to be limited to verifiable instances, such as recorded speeches or publications, as there would be no way to evaluate the amount of times it's done in personal conversations.

25
  • 10
    Not intended to be a factual statement
    – Monkey Tuesday
    Commented May 22, 2011 at 22:31
  • 12
    Not intended to be a factual statement is right up there with the Texas Board of Education textbooks! The Chinese, Indians, and every other country in the world must be chortling and just saying, "Excellent!" as they watch the US slide into Idiocracy.
    – Larian LeQuella
    Commented May 23, 2011 at 1:33
  • 3
    @oddthinking An ad hominem attack implies that it is either Jon Kyl's claims or credibility in question. I assure you, they are not, he is referenced only because his behavior is a famous and recent example which illustrates the larger question about the prevalency of fabricted statistics. Was it an insulting thing to say? Absolutely. Was it necessary? Probably not. But ad hominem attack? Nope.
    – Monkey Tuesday
    Commented May 23, 2011 at 4:38
  • 5
    @Rusty intellectual dishonesty is essentially the advocation of a position known to be false. The other gentleman claimed that use of Plan B causes an abortion. This is false. Plan B is not an abortificant. But because he is Catholic the use of Plan B is, in his opinion, the same as having an abortion. However, the use of the ordinary birth control pill and the mirena iud are only 'problematic' for him, and not considered abortion to him despite the fact it's the same chemical compound.
    – Darwy
    Commented May 23, 2011 at 8:34
  • 5
    12 out of 8 people don't understand statistics ;)
    – Oliver_C
    Commented May 23, 2011 at 13:04

1 Answer 1

Reset to default
12
votes

Noted skeptic, Ben Goldacre, took part in a published study recently. It doesn't directly answer your stated question, but it does address some of the concern behind it.

Benjamin E.J. Cooper, William E. Lee, Ben M. Goldacre, Thomas A.B. Sanders, "The quality of the evidence for dietary advice given in UK national newspapers", Public Understanding of Science, April 2001, doi: 10.1177/0963662511401782

Goldacre described the research in his Guardian column.

They didn't look at all statistics quoted. They restricted their scope to health claims (note: not just statistics) made by newspapers. They diligently tried to find evidence to support them.

(Can someone please invite those researchers to come to this site next, and use those same skills?)

Goldacre summarised in his column:

Here's what we found: 111 health claims were made in UK newspapers over one week. The vast majority of these claims were only supported by evidence categorised as "insufficient" (62% under the WCRF system). After that, 10% were "possible", 12% were "probable", and in only 15% was the evidence "convincing". Fewer low quality claims ("insufficient" or "possible") were made in broadsheet newspapers, but there wasn't much in it.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .