Take the 2-minute tour ×
Skeptics Stack Exchange is a question and answer site for scientific skepticism. It's 100% free, no registration required.

There exists a widely held (and frequently asserted) belief that there is a very big (relatively to other professions) discrepancy between the top tier software developers and the non-top-tier.

Examples from extremely reputable people include:

  • Paul Graham (in a bunch of his writings: [1] , [2] )

  • Joel Spolsky: ( [1] , [2] )

    ... of superstars that produce ten times as much as the merely brilliant software developers.

Are there any studies supporting this? (e.g. that a "superstar" programmer would be 10x more productive than merely good programmer or 30-100x than average programmer respectively)

Just to clarify - I'm not asking if Joel was correct in naming 100x more productive developers "superstars" - that's his definition. I'm asking if there are enough such 100x more productive developers (100x on average) to warrant a separate class.

share|improve this question
2  
Saeed Dehnadi and Richard Bornat: The camel has two humps and critique by Jeff Attwood: Separating programming sheep from non programming goats, which mostly focuses on the first programming class, rather than the more seasoned programmers this question is focused on. –  TokenMacGuy Jan 15 '12 at 15:54
1  
Trying to quantify it is flawed. How do you measure programmers productivity? If programmer A finishes product in 5 days, and programmer B also finishes product in 5 days, but he has brilliant idea on how to optimize the process, which also makes final product 1000 times faster, then how much "more productive" is programmer B, than programmer A? –  vartec Jan 16 '12 at 14:54
    
@Vartec - I wouldn't say that "trying to quantify" is "flawed". It's difficult to quatify, with most of the trivial/easy methodologies easily gamed or simply poor for the reasons you stated and more. But that doesn't mean that it's impossible to quantify, or that the desire to quantify in and out of itself is flawed. –  DVK Jan 16 '12 at 15:01
2  
@DVK: still, it's flawed, as 10 interns will be way more effective making cookie-cut form interfaces, than 1 brilliant programmer at same job. In fact even one inter will be probably more effective. On the other hand 1 brilliant programmer will be infinitely more effective at designing complex, scalable system, as 10 clueless interns will just fail to deliver at all. –  vartec Jan 16 '12 at 15:48
3  
I seem to remember first encountering this statistic in the Psychology of Computer Programming by Gerald M. Weinberg. If I remember correctly it was based on research at IBM. (It was a library copy from perhaps 25 years ago, so I don't have it to hand). It may also have referred to fully-debugged, documented code. (Remember somebody introducing a bug can cause days of work for other people trying to find it - the worst programmers in my experience actually have negative productivity!) –  MZB Jan 17 '12 at 18:39
show 5 more comments

2 Answers

up vote 11 down vote accepted

Common source for these claims is Peopleware. The authors claim that while conducting programming competitions called Coding War Games during the 1980s they found that:

Three rules of thumb seem to apply whenever you measure variations in performance over a sample of individuals:

  • Count on the best people outperforming the worst by about 10:1.
  • Count on the best performer being about 2.5 times better than the median performer.
  • Count on the half that are better-than-median performers outdoing the other half by more than 2:1.

More modern source is Steve McConnell's article in "What Does 10x Mean? Measuring Variations in Programmer Productivity" Making Software where the author refers to several other sources that are mostly from the 1980s. If someone wants to hunt these studies down, the sources are:

  • [Boehm et al. 2000] Boehm, Barry, et al. 2000. Software Cost Estimation with Cocomo II. Boston: Addison-Wesley.
  • [Boehm and Papaccio 1988] Boehm, Barry W., and Philip N. Papaccio. 1988. Understanding and Controlling Software Costs. IEEE Transactions on Software Engineering 14(10): 1462–1477.
  • [Curtis 1981] Curtis, Bill. 1981. Substantiating Programmer Variability. Proceedings of the IEEE 69(7): 846.
  • [DeMarco and Lister 1985] DeMarco, Tom, and Timothy Lister. 1985. Programmer Performance and the Effects of the Workplace. Proceedings of the 8th International Conference on Software Engineering: 268–272.
  • [Mills 1983] Mills, Harlan D. 1983. Software Productivity. Boston: Little, Brown.
  • [Card 1987] Card, David N. 1987. A Software Technology Evaluation Program. Information and Software Technology 29(6): 291–300.
  • [Curtis et al. 1986] Curtis, Bill, et al. 1986. Software Psychology: The Need for an Interdisciplinary Program. Proceedings of the IEEE 74(8): 1092–1106.
  • [Valett and McGarry 1989] Valett, J., and F.E. McGarry. 1989. A Summary of Software Measurement Experiences in the Software Engineering Laboratory. Journal of Systems and Software 9(2): 137–148.

For interesting discussion on programming ability related to salary check Stackoverflow podcast #77.

share|improve this answer
1  
Coding wars is a very good data source, though probably only for upper end of the spectrum. I don't think mediocre programmers enrolled, though not 100% sure (was before my time - the only things I ever competed in were ACM contests). –  DVK Jan 16 '12 at 14:07
2  
@DVK - I do not accept that what was true in the days of 8 bit processors/compilers and < 256k ram is true today. The productivity tools that exist now enable even mediocre programmers to produce effective code that executes complex functionality relatively easily. In the 80's we basically had macros for our text editors and cheat sheets. –  Chad Jan 17 '12 at 14:28
3  
@Chad - it's not about the tools. It's about understanding and designing abstractions. Src: Joel Spolsky :) –  DVK Jan 17 '12 at 14:41
1  
@Chad - I donno... so far, only ~30% or less of my own SO questions were answered to my satisfaction :)))) Most were not useful/helpful at all. And I would posit that the tools give MORE of an edge to a good programmer than a bad one, introducing extra leverage. Just my theory (my first "tool" was a Sinclair Z80 Basic :) –  DVK Jan 17 '12 at 14:51
1  
Laurent Bossavit criticizes McConnell's citations here: morendil.github.com/folklore.html McConnell responds here: goo.gl/AEpul (FWIW, I've read every one of the sources and feel that all have questionable aspects: students, very short durations, "apples to oranges" language levels, etc.) –  Larry OBrien Jan 18 '12 at 18:22
show 2 more comments

Here is a well documented study. Individual productivity was not the goal of this paper, but there is data that's related to your question. A copy of the paper can be found here

On page 22, there's a box graph of hours needed to complete the program by language. If we assume programmer productivity to be time to complete a given programming task, then we have a maximum range of productivity of ~4 hours - 63 hours within Java. We can say that the best programmer did perform 10 times faster than the worst.

Original paper:

An empirical comparison of C, C++, Java, Perl, Python, Rexx, and Tcl for a search/string-processing program by Lutz Prechelt (2000)

If you were interested, the paper includes the raw data, and you can perform your own analysis on the distribution. The distribution would tell you approximately how many programmers you would need to say that the best programmer is 100x more productive than the worst.

share|improve this answer
2  
Prechelt is a good researcher. I would caution, though, that a developer who excels at one task might not do so well consistently -- familiarity with the task versus pursuing dead ends seems to me an important part of the job. And perhaps over many tasks developers regress toward the mean. –  Larry OBrien Mar 16 at 19:25
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.