25

I was reading through post-debate comments made by prominent writers and the such, and came across one who linked the Washington Post (who linked Politifact) in saying that of the statements made by Donald Trump that the site fact-checked, only 22% were rated 'true'. (78% were rated 'mostly false' or worse.).

A fact checker looked into 158 things Donald Trump said. 78 percent were false.

I'm a staunch moderate, but after the debate, I thought I should learn a bit more about the platforms of each party, so I have to know.

Are these statistics correct?

30

TL;DR: That's not how fact checks work.

The stats

Politifact has a Trump page which as of right now (October 4th, 2016) says

True           (4%)(12)
Mostly True   (11%)(31)
Half True     (14%)(39)
Mostly False  (18%)(48)
False         (35%)(96)
Pants on Fire (18%)(48)

If you do the math, this gives 71% Mostly False or worse (False and Pants on Fire) versus 29% Half True or better (Mostly True or True).

So on the face of it, the 22% statistic is currently wrong, although it may have been facially correct at the time it was originally posted.

What is truth?

This hides a bigger problem. Politifact in particular and fact checkers in general do not measure truth. They measure factual inaccuracy. Some factual inaccuracy may be lies. That's essentially the purpose of the Pants on Fire rating. That's when a politician says something knowing that it is not true.

Most factual inaccuracy is just that though. Someone says something that is an exaggeration or misleadingly stated. For example, this question title currently says: "Did Donald Trump only tell the truth in 22% of his fact-checked statements?" That is a misleading restatement of "only 22% were rated 'true'. (78% were rated 'mostly false' or worse.)."

Mostly False used to be called Barely True. Such statements contain a kernel of truth. From the definition of Mostly False:

The statement contains an element of truth but ignores critical facts that would give a different impression.

So Trump told some truth in every Mostly False claim. If he hadn't, it would have been ruled False or Pants on Fire instead. Adding Mostly False to the true bin would increase the statistic to 47%.

An example

The way to avoid Mostly False ratings is to be careful in what one says. For example:

Says "no, you’re wrong," responding to the assertion that stop-and-frisk was ruled unconstitutional in New York.

But Politifact says:

Trump has a point that a judge’s ruling in a 2013 case did not declare stop-and-frisk as a general practice unconstitutional in New York. Stop-and-frisk is still legal and still takes place in New York and across the country.

They don't claim that he wasn't truthfully describing the constitutionality of stop-and-frisk. They claim that there is an alternative interpretation of his statement (that he may not have intended) that is false:

But it also isn’t quite correct for Trump to call Holt’s claim "wrong" because that implies there was no finding of unconstitutionality in New York’s practices or that Scheindlin’s ruling was tossed out, when really it still stands.

So if Trump had said what they said, e.g.

Constitutional stop-and-frisk is legal across America

He would have gotten a True rating for essentially the same point. Fact checks frequently turn on small semantic differences like this. Note that I found this one by picking the first national claim on Trump's Mostly False page (at this moment; likely to change in the future). That's why Mostly True, Half True, and Mostly False exist: to measure small semantic differences between claims.

Bias

US News and World Report says Politifact is biased against Republicans. As do others. And others. Or see an analysis of Democrats and Republicans with different ratings for similar statements.

  • 2
    +1 for a great answer. This is really helpful to me, as I sift through all of the articles from political pundits, it's good to see such a well thought out answer. – Anoplexian Oct 5 '16 at 16:02
  • 8
    This answer is play on words. And misses the real problem - anyone's fact checked claims will be likely to be unfavorable, because people don't fact-check obvious truths, but they will fact-check obvious lies. And I don't see how is it relevant that some news pages say that polifact is biased against republicans. Were any of their fact-checks incorrect? – Tomáš Zato Dec 5 '16 at 2:07
  • Re bias: Steven Colbert (a comedian of course) said "reality has a well-known liberal bias" en.wikiquote.org/wiki/Stephen_Colbert – Peter M. Dec 5 '16 at 20:41
  • 7
    Which wouldn't explain why Bernie Sanders could get a Mostly True and Donald Trump could get a Mostly False on the same statement. A reality bias doesn't explain that--only a bias in the checker. And bias matters because if you move all the statements from one bucket to another, it changes the statistic that the claim measures. – Brythan Dec 5 '16 at 23:10
  • 2
    @Brythan For what it's worth, the Tampa Bay times (owner of PolitiFact, see the copyright text at the very bottom of PolitiFact.com) endorsed Hillary Clinton for president last October. – user2428118 Jan 24 '17 at 22:51

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .