13

Does the 1999 paper by Kruger and Dunning "Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments" reliably prove that incompetents have a consistently higher self-appraisal than competents?

I'm accepting that the gap between self-appraisal and reality is greater for incompetents than for competents (the paper discusses the potential for a regression effect in their statistical analysis), and that most people think they are above average ("Illusory superiority" or the "Lake Woebegone effect"), but the paper says that more than that is going on. It describes Charles Darwin as saying

"ignorance more frequently begets confidence than does knowledge"

and this study has been cited by the general public:

@gilles that's kind of intentional; we basically don't want these users. en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect – Jeff Atwood♦ May 15 at 20:03

(As far as I can tell, Atwood was implying that incompetent users are more likely to think that there's an error on the Stack Exchange servers than competent users)

Has the methodological rigour, and the relevance, of the paper been generally accepted?

Also, is their claim that incompetents have a higher self-appraisal than competents been treated by the scientific community as strongly confirmed, or is there still doubt about this claim within the mainstream?

If the claims are true, how come there wasn't a negative correlation between actual ability and perceived ability in figures 1, 3 and 4 in the 1999 paper? (PDF)

Figure 1

enter image description here

Figure 3

enter image description here

Figure 4

enter image description here

  • Isn't this similar to the paper "Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments and inflated worth to the community at large inside the civil service" – Hairy May 20 '11 at 10:05
  • Are you talking about the figures from the original study? What do you mean there isn't a negative correlation? As far as I can tell, there is, and the results align perfectly with the conclusions...? – David Hedlund May 20 '11 at 10:28
  • 3
    @Andrew: the absolute value doesn't decrease, but it decreases relative to the actual ability. Those who are in the top quartile perceive their ability as lower than their actual test scores imply. So there's definitely a negative correlation in confidence, which is what the Darwin quote is saying, and which is, from what I can gather, all that is being said at all...? – David Hedlund May 20 '11 at 11:39
  • 1
    In fact, even with the actual figures, we can only conclude the presence of weak correlations in Figure 1 and 4 (and its absence in Figure 3). To determine whether people whose skill level is in the lowest quartile have a tendency to feel superior to the more competent, a scatter plot would be needed. – user2547 May 21 '11 at 19:48
  • 1
    A negative correlation would imply that the incompetent thought that they were superior to the experts, which would be a very extreme finding. The paper (as I understand it) merely says that the incompetent over-estimate their ability (so their true skill is on average less than their self-assessment) and that the highly competent tend to under-estimate their ability (so their true skill score is higher than their self assessment). This is indeed what we see in figures 1, 3 & 4, and can be the case without the incompetent's self-assessed scores being higher than those of the experts. – Dikran Marsupial Mar 2 '15 at 21:29
3

I think the Wikipedia article covers the support for this phenomenon adequately in its "supporting studies" section. There's no reason to doubt that it's a real effect given the number of confirmations.

As an example, Burson, Larrick, and Klayman (2006) set out to test the effect, asking whether people were judging task difficulty instead of their own ability. They performed similar tests but with varying difficulties and reproduced the original results, i.e., that people are poor at estimating their relative ability (Figure 4):

Burson et al, Figure 4

The gold standard of rigor in sciences is ability to replicate findings (that's the point of being rigorous!), so in this regard the paper passes: even if there were some methodological problems (it's often really hard to tell), the results seem sound.

With regard to confidence vs. ability, the Burson paper shows conditions (such as the "Hard Trivia" line above where the best performers have the most inaccurate estimate of confidence. This argues against Kruger & Dunning's hypothesis that higher ability is (necessarily) linked to more accurate judgments of self-ability.

Finally, you ask whether that the authors' claim that "incompetents have a higher self-appraisal than competents" is tested and accepted. That claim is tested and refuted by the authors, by Burson et. al, and others: correlations with performance are typically positive or non-existent, not negative. But the authors do not make the claim (quote from Darwin notwithstanding)! Here are a selection of quotes from Kruger and Dunning about what they do claim:

Prediction 1 Incompetent individuals, compared with their more competent peers, with dramatically overestimate their ability and performance compared to objective criteria. (p. 1112)

Emphasis mine: the claim is that they overestimate the most, not that their estimates are even higher than the estimates of high-performers!

To be sure, they [the poor-performers] had an inkling that they were not as talented in this domain as were participants in the top quartile, as evidenced by the significant correlation between perceived and actual ability.

Again, they agree that poor performers do not give estimates that outstrip the estimates of top performers.

Bottom-quartile participants were less successful than were top-quartile participants in the metacognitive tasks of discerning what one has answered correctly vs. incorrectly

Language like the above is seen throughout: not a statement that bottom-quartile estimates themselves higher than top-quartile estimates themselves, but that bottom-quartile has got it really badly wrong.

Other studies (e.g. Burson et al.) use similar language and show similar effects (though there is not solid agreement on the causes of the effects).

So that claim isn't made by the authors (at least in this paper), and if someone in the general public were to make such a claim they should be aware that it has been (weakly) refuted by (at least some) studies of these effects.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .