Does "startup employee effectiveness follow a power law"?

Question

One of the main assertions of this blog post is that, "Startup employee effectiveness follows a power law" rather than a normal distribution.

Much like startup performance follows a power law, so do startup employees. The most effective employees create 20x more leverage than an average employee. This is not true in an efficiency company — the best employees might work 2x faster than their peers. But in a high-leverage startup like ours, the effectiveness gap between employees can be multiple orders of magnitude.

Our minds find it easier to think in terms of efficiency and normal distributions than leverage and power law distributions. So we mentally squash the employee power law curve into a normal distribution curve. We underestimate the most effective employees and overestimate the ineffective ones.

Aside from the fact that this is a gross over-simplification of productivity within the context of development, I think the reason why we assume a normal distribution is because thats how any random variable with a population works (I'm assuming within the population of professional developers programmers).

I've also similar claims to this, in the form of the, "10x engineer".

Can anyone provide any research to back up these assertions? Or perhaps someone with a better understanding of stats help?

Answer 1

I've found an interview with Laurent Bossavit, were he discusses his book "The Leprechauns of Software Engineering". He makes that claim that the "10x Programmer" is for small sample sets with very old data. Many of the references are themselves to just copies of the claim.

When I looked into it, what was advanced as evidence for those claims, what I found was not really what I had expected, what you think would be the case for something people say, and what you think is supported by tens of scientific studies and research into software engineering. In fact what I found when I actually investigated, all the citations that people give in support for that claim, was that in many cases the research was done on very small groups and not extremely representative, the research was old so this whole set of evidence was done in the seventies, on programs like Fortran or COBOL and in some cases on non-interactive programming, so systems where the program was input, you get results of the compiling the next day.

And:

And also many of the papers and books that were pointed to were not properly scientific papers. They were opinion pieces or books like Peopleware, which I have a lot of respect for but it’s not exactly academic.

I haven't got a copy of the book, but he says that he's actually looked at the original evidence.

I've also found many articles published last year that makes claims about the benefits of eliminating toxic workers see here

The results were startling. A top 1 percent superstar—a very rare high performer—brings an extra $5,300 in value by doing more work than an average employee does. Replacing a toxic worker with an average one creates an estimated $12,800 in cost savings over the same period by reducing the cost of turnover around that toxic worker. Similarly, replacing a toxic worker yielded almost four times the value of hiring a top 10 percent performer.