23

This is really a followup question to Is programming in Python faster than in C, C++ or Java? that I also posted on programmers.SE. The first comment seem to challenge the very idea that programmer productivity can be measured to begin with:

Well, since you've restricted the set of possible answers, I just dare a comment by asking another question which should be answered first (imho): Is there a reliable and estabilished metrics for measuring the "productivity of a programmer"?

He later goes on to argue that there are so many ways to measure productivity, but they are only applicable to whatever narrow thing they were analyzing:

Yes, there just "too many" ways which, from my point of view, very often contradict each other. All articles, books, blogs, talks.. I have read or heard with regard to this topic were more or less subjective or narrowed to a very specific set of problems making the results hardly applicable.

Further Joel Spoolsky in 2005 wrote a post called Hitting the High Notes:

Let's start with plain old productivity. It's rather hard to measure programmer productivity; almost any metric you can come up with (lines of debugged code, function points, number of command-line arguments) is trivial to game, and it's very hard to get concrete data on large projects because it's very rare for two programmers to be told to do the same thing.

He then reference a CS class where they measured time and quality of software with great variance, after which Joel concluded:

There's just nothing to see here, and that's the point. The quality of the work and the amount of time spent are simply uncorrelated.

Has this issue been tackled by researchers with repeatable results, or do we have no idea how to reliably measure productivity of programmers as a whole, beyond the opinions of individual programmers common-sense and experience?

Guidance towards answering the question

Productivity: (economics) the ratio of the quantity and quality of units produced to the labor per unit of time

So by the very definition of the word productivity, you can arrive at the same level of productivity, but at different points of the spectrum (several units produced, quality of fewer units). As such it's important to declare how productivity changed, and why we think it increased instead of decreased, on the whole. Also if productivity goes up in all imaginable metrics then by any definition, that's being more productive.

Wikipedia on programmer productivity suggest the following dimensions:

  • Amount of code that can be created
  • Amount of code that can be maintained
  • Detecting and avoiding errors
  • Software cost estimation

Reliable metrics in this case means:

If different groups of people do the same change, they will see the same results once controlled for external factors.

Assumption of no malice: The programmers being measured aren't trying to intentionally circumvent the metric, by for instance adding extra lines of code when the metric is lines-of-code.

Related on programmers.SE:

  • 6
    To me, the problem with LoC as a means of measuring productivity is not that devs will try to add superfluous lines, but rather, that amount of lines produced simply does not correlate with amount of productivity. I can be hugely productive and still leave every file at a net decrease of LoC. It's a poor measure of productivity, and that's that. I realize that your question wasn't specifically about lines of code, though, hence the comment. – David Hedlund Apr 19 '11 at 6:30
  • 1
    @David Hedlund - I agree. Though on one hand if I produce a million lines of code in the same time-span as I do 1000, I'm pretty sure that it's very unlikely I have been more productive in the latter case. I'm sure LoC measures something. – Kit Sunde Apr 19 '11 at 6:35
  • 3
    Yes. If you have produced 1000 codes, chances are you have worked smarter, not harder :-P – Lagerbaer Apr 19 '11 at 20:37
  • 1
    I think the assumption of no malice is going to invalidate any answers you get - unless you're keeping your metrics system secret (and, hey, programmers are probably smart enough that a few of them could get together and reverse-engineer the metrics system from the bonuses they got), introducing such a system will change the culture such that fitting in requires working to the metrics. I'll also point to another Spolsky article. – Joel Rein May 11 '11 at 0:55
  • 5
    @KitSunde: If you can reduce 10,000 lines of code to a single math equation, you've probably just made the program run exponentially faster. Doing that kind of work cannot be measured, but it's damn fine and brilliant work. Which is why it's so important to have managers that have some semblance of a clue about what the workers are doing. – Ernie Nov 18 '11 at 18:53

You must log in to answer this question.

Browse other questions tagged .