28

There exists a widely held (and frequently asserted) belief that there is a very big (relatively to other professions) discrepancy between the top tier software developers and the non-top-tier.

Examples from extremely reputable people include:

  • Paul Graham (in a bunch of his writings: [1] , [2] )

  • Joel Spolsky: ( [1] , [2] )

    ... of superstars that produce ten times as much as the merely brilliant software developers.

Are there any studies supporting this? (e.g. that a "superstar" programmer would be 10x more productive than merely good programmer or 30-100x than average programmer respectively)

Just to clarify - I'm not asking if Joel was correct in naming 100x more productive developers "superstars" - that's his definition. I'm asking if there are enough such 100x more productive developers (100x on average) to warrant a separate class.

Improve this question
11
  • 2
    Saeed Dehnadi and Richard Bornat: The camel has two humps and critique by Jeff Attwood: Separating programming sheep from non programming goats, which mostly focuses on the first programming class, rather than the more seasoned programmers this question is focused on.
    – SingleNegationElimination
    Jan 15, 2012 at 15:54
  • 2
    Trying to quantify it is flawed. How do you measure programmers productivity? If programmer A finishes product in 5 days, and programmer B also finishes product in 5 days, but he has brilliant idea on how to optimize the process, which also makes final product 1000 times faster, then how much "more productive" is programmer B, than programmer A?
    – vartec
    Jan 16, 2012 at 14:54
  • 4
    @DVK: still, it's flawed, as 10 interns will be way more effective making cookie-cut form interfaces, than 1 brilliant programmer at same job. In fact even one inter will be probably more effective. On the other hand 1 brilliant programmer will be infinitely more effective at designing complex, scalable system, as 10 clueless interns will just fail to deliver at all.
    – vartec
    Jan 16, 2012 at 15:48
  • 3
    I seem to remember first encountering this statistic in the Psychology of Computer Programming by Gerald M. Weinberg. If I remember correctly it was based on research at IBM. (It was a library copy from perhaps 25 years ago, so I don't have it to hand). It may also have referred to fully-debugged, documented code. (Remember somebody introducing a bug can cause days of work for other people trying to find it - the worst programmers in my experience actually have negative productivity!)
    – MZB
    Jan 17, 2012 at 18:39
  • 1
    Is it possible to measure the productivity of lone programmers as individuals at all at least for big modern software projects? Don't you have to evaluate the whole team? Even an individually productive programmer can be brought low by a poor team, poor leadership or poor coordination.
    – matt_black
    Jan 17, 2012 at 21:31

2 Answers 2

Reset to default
19

Common source for these claims is Peopleware. The authors claim that while conducting programming competitions called Coding War Games during the 1980s they found that:

Three rules of thumb seem to apply whenever you measure variations in performance over a sample of individuals:

  • Count on the best people outperforming the worst by about 10:1.
  • Count on the best performer being about 2.5 times better than the median performer.
  • Count on the half that are better-than-median performers outdoing the other half by more than 2:1.

More modern source is Steve McConnell's article in "What Does 10x Mean? Measuring Variations in Programmer Productivity" Making Software where the author refers to several other sources that are mostly from the 1980s. If someone wants to hunt these studies down, the sources are:

  • [Boehm et al. 2000] Boehm, Barry, et al. 2000. Software Cost Estimation with Cocomo II. Boston: Addison-Wesley.
  • [Boehm and Papaccio 1988] Boehm, Barry W., and Philip N. Papaccio. 1988. Understanding and Controlling Software Costs. IEEE Transactions on Software Engineering 14(10): 1462–1477.
  • [Curtis 1981] Curtis, Bill. 1981. Substantiating Programmer Variability. Proceedings of the IEEE 69(7): 846.
  • [DeMarco and Lister 1985] DeMarco, Tom, and Timothy Lister. 1985. Programmer Performance and the Effects of the Workplace. Proceedings of the 8th International Conference on Software Engineering: 268–272.
  • [Mills 1983] Mills, Harlan D. 1983. Software Productivity. Boston: Little, Brown.
  • [Card 1987] Card, David N. 1987. A Software Technology Evaluation Program. Information and Software Technology 29(6): 291–300.
  • [Curtis et al. 1986] Curtis, Bill, et al. 1986. Software Psychology: The Need for an Interdisciplinary Program. Proceedings of the IEEE 74(8): 1092–1106.
  • [Valett and McGarry 1989] Valett, J., and F.E. McGarry. 1989. A Summary of Software Measurement Experiences in the Software Engineering Laboratory. Journal of Systems and Software 9(2): 137–148.

For interesting discussion on programming ability related to salary check Stackoverflow podcast #77.

Improve this answer
10
  • 1
    Coding wars is a very good data source, though probably only for upper end of the spectrum. I don't think mediocre programmers enrolled, though not 100% sure (was before my time - the only things I ever competed in were ACM contests).
    – user5341
    Jan 16, 2012 at 14:07
  • 2
    @DVK - I do not accept that what was true in the days of 8 bit processors/compilers and < 256k ram is true today. The productivity tools that exist now enable even mediocre programmers to produce effective code that executes complex functionality relatively easily. In the 80's we basically had macros for our text editors and cheat sheets.
    – Chad
    Jan 17, 2012 at 14:28
  • 4
    @Chad - it's not about the tools. It's about understanding and designing abstractions. Src: Joel Spolsky :)
    – user5341
    Jan 17, 2012 at 14:41
  • 2
    @Chad - I donno... so far, only ~30% or less of my own SO questions were answered to my satisfaction :)))) Most were not useful/helpful at all. And I would posit that the tools give MORE of an edge to a good programmer than a bad one, introducing extra leverage. Just my theory (my first "tool" was a Sinclair Z80 Basic :)
    – user5341
    Jan 17, 2012 at 14:51
  • 1
    @DVK - But this is talking about quantity not quality. With VS and asp.net an amateur can whip up a fairly functional site with database connectivity quickly and by writing very little if any code that hits a database and has a fairly secure login functionality. To me the senior knows how to go beyond that and fix the holes those types of solutions leave. The superstar creates Google/Facebook etc. The superstar does not work for your company you work for him.
    – Chad
    Jan 17, 2012 at 17:58
13

Here is a well documented study. Individual productivity was not the goal of this paper, but there is data that's related to your question. A copy of the paper can be found here

On page 22, there's a box graph of hours needed to complete the program by language. If we assume programmer productivity to be time to complete a given programming task, then we have a maximum range of productivity of ~4 hours - 63 hours within Java. We can say that the best programmer did perform 10 times faster than the worst.

Original paper:

An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl for a search/string-processing program by Lutz Prechelt (2000)

If you were interested, the paper includes the raw data, and you can perform your own analysis on the distribution. The distribution would tell you approximately how many programmers you would need to say that the best programmer is 100x more productive than the worst.

Improve this answer
1
  • 4
    Prechelt is a good researcher. I would caution, though, that a developer who excels at one task might not do so well consistently -- familiarity with the task versus pursuing dead ends seems to me an important part of the job. And perhaps over many tasks developers regress toward the mean.
    – Larry OBrien
    Mar 16, 2014 at 19:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .