29

This is really a followup question to Is programming in Python faster than in C, C++ or Java? that I also posted on programmers.SE. The first comment seem to challenge the very idea that programmer productivity can be measured to begin with:

Well, since you've restricted the set of possible answers, I just dare a comment by asking another question which should be answered first (imho): Is there a reliable and established metrics for measuring the "productivity of a programmer"?

He later goes on to argue that there are so many ways to measure productivity, but they are only applicable to whatever narrow thing they were analyzing:

Yes, there just "too many" ways which, from my point of view, very often contradict each other. All articles, books, blogs, talks.. I have read or heard with regard to this topic were more or less subjective or narrowed to a very specific set of problems making the results hardly applicable.

Further Joel Spolsky in 2005 wrote a post called Hitting the High Notes:

Let's start with plain old productivity. It's rather hard to measure programmer productivity; almost any metric you can come up with (lines of debugged code, function points, number of command-line arguments) is trivial to game, and it's very hard to get concrete data on large projects because it's very rare for two programmers to be told to do the same thing.

He then reference a CS class where they measured time and quality of software with great variance, after which Joel concluded:

There's just nothing to see here, and that's the point. The quality of the work and the amount of time spent are simply uncorrelated.

Has this issue been tackled by researchers with repeatable results, or do we have no idea how to reliably measure productivity of programmers as a whole, beyond the opinions of individual programmers common-sense and experience?

Guidance towards answering the question

Productivity: (economics) the ratio of the quantity and quality of units produced to the labor per unit of time

So by the very definition of the word productivity, you can arrive at the same level of productivity, but at different points of the spectrum (several units produced, quality of fewer units). As such it's important to declare how productivity changed, and why we think it increased instead of decreased, on the whole. Also if productivity goes up in all imaginable metrics then by any definition, that's being more productive.

Wikipedia on programmer productivity suggest the following dimensions:

  • Amount of code that can be created
  • Amount of code that can be maintained
  • Detecting and avoiding errors
  • Software cost estimation

Reliable metrics in this case means:

If different groups of people do the same change, they will see the same results once controlled for external factors.

Assumption of no malice: The programmers being measured aren't trying to intentionally circumvent the metric, by for instance adding extra lines of code when the metric is lines-of-code.

Related on programmers.SE:

  • 1
    Please do not add answers in comments. Please do not add theoretical answers or answers which contain your point of view. Find studies on the matter and tell us their findings. I've removed two answers on the basis that they are not based on evidence and highly upvoted. – Sklivvz Feb 28 at 14:26
  • 2
    Lines of code, even if the programmer is not maliciously adding more, is ridiculous. Simple changes in style such as putting braces on their own line, often done automatically by the IDE or a code formatter, game that metric. There is also the issue of code re-use and libraries that make even measuring lines of code per programmer difficult. – dont_shog_me_bro Mar 1 at 9:50
  • 2
    Technical debt also makes measurements of time largely worthless. Yes, you can hack out a working lump of code in a few hours, but if you spend a few days you can build a flexible and extensible platform to build on that saves time later. – dont_shog_me_bro Mar 1 at 9:51
  • 2
    Debugging is also missing for most of these metrics. Measuring bugs is not really satisfactory. You can assign vague difficulty ratings to bugs, but that's self defeating because if a bug takes a week to fix you rank it "extremely difficult" and it looks like a productive week... Even if it was actually just that the programmer didn't know how to use the debugging tools properly. – dont_shog_me_bro Mar 1 at 9:51
  • 1
    "Lines of code" is an utterly useless metric - even ignoring intentional or accidental gaming. I spent three days of the last work week rewriting a section of our program code in order to work around a bug in a third party library. Not only does the new code work around the bug, it is about half the length of the old method and avoids the creation a file as an intermediate step. Do I get negative productivity for removing code? – JRE Mar 1 at 12:19

You must log in to answer this question.

Browse other questions tagged .