Take the 2-minute tour ×
Skeptics Stack Exchange is a question and answer site for scientific skepticism. It's 100% free, no registration required.

enter image description here

Comedian Stephen Wright once joked that 42.7% of all statistics are made up on the spot.

Recently, Arizona Senator and notably stubborn Jon Kyl provided us with the most famous example of just such behavior.

He can be seen here, stating that well over 90% of what planned parenthood does is abortions, despite the fact that the real amount is 3%. Later, he attempted to claim that his words were "not intended to be a factual statement" once it became clear he had just made up the 90% figure.

enter image description here

In a perfect world, all claims would be appropriately and accurately referenced, but we all know this just simply isn't the case, especially when it comes to politicians, the media, and those pushing an agenda dishonestly.

Has there ever been any scientific examination into how often quoted statistics are just made up? Do we honestly have any way to figure this out?

I assume any study of this would have to be limited to verifiable instances, such as recorded speeches or publications, as there would be no way to evaluate the amount of times it's done in personal conversations.

share

locked by Sklivvz Jul 12 at 9:50

This question exists because it has historical significance, but it is not considered a good, on-topic question for this site, so please do not use it as evidence that you can ask similar questions here. This question and its answers are frozen and cannot be changed. More info: help center.

10  
Not intended to be a factual statement –  Monkey Tuesday May 22 '11 at 22:31
12  
Not intended to be a factual statement is right up there with the Texas Board of Education textbooks! The Chinese, Indians, and every other country in the world must be chortling and just saying, "Excellent!" as they watch the US slide into Idiocracy. –  Larian LeQuella May 23 '11 at 1:33
3  
@oddthinking An ad hominem attack implies that it is either Jon Kyl's claims or credibility in question. I assure you, they are not, he is referenced only because his behavior is a famous and recent example which illustrates the larger question about the prevalency of fabricted statistics. Was it an insulting thing to say? Absolutely. Was it necessary? Probably not. But ad hominem attack? Nope. –  Monkey Tuesday May 23 '11 at 4:38
5  
@Rusty intellectual dishonesty is essentially the advocation of a position known to be false. The other gentleman claimed that use of Plan B causes an abortion. This is false. Plan B is not an abortificant. But because he is Catholic the use of Plan B is, in his opinion, the same as having an abortion. However, the use of the ordinary birth control pill and the mirena iud are only 'problematic' for him, and not considered abortion to him despite the fact it's the same chemical compound. –  Darwy May 23 '11 at 8:34
5  
12 out of 8 people don't understand statistics ;) –  Oliver_C May 23 '11 at 13:04

1 Answer 1

up vote 12 down vote accepted

Noted skeptic, Ben Goldacre, took part in a published study recently. It doesn't directly answer your stated question, but it does address some of the concern behind it.

Benjamin E.J. Cooper, William E. Lee, Ben M. Goldacre, Thomas A.B. Sanders, "The quality of the evidence for dietary advice given in UK national newspapers", Public Understanding of Science, April 2001, doi: 10.1177/0963662511401782

Goldacre described the research in his Guardian column.

They didn't look at all statistics quoted. They restricted their scope to health claims (note: not just statistics) made by newspapers. They diligently tried to find evidence to support them.

(Can someone please invite those researchers to come to this site next, and use those same skills?)

Goldacre summarised in his column:

Here's what we found: 111 health claims were made in UK newspapers over one week. The vast majority of these claims were only supported by evidence categorised as "insufficient" (62% under the WCRF system). After that, 10% were "possible", 12% were "probable", and in only 15% was the evidence "convincing". Fewer low quality claims ("insufficient" or "possible") were made in broadsheet newspapers, but there wasn't much in it.
share

Not the answer you're looking for? Browse other questions tagged or ask your own question.