I was torn between posting this on Cross Validated, User Experience, or here, but here goes:
Many have heard about how Google, not wanting to rely on the hunches of designers, tested 41 shades of blue for their links and found that the best performing shade led to $US200M in revenue a year.
Google's commitment to data-driven decisions is well reported, and the company has been ridiculed for the "50 shades of blue" episode, when then Google executive Marissa Meyer led a project testing the impact of using different coloured links in ads.
But a new insight proves that the company significantly benefitted from the experiment, to the tune of $200m.
I'm not skeptical so much about whether the tests actually happened, or even if Google projected that the change would generate that much extra revenue, but the counter-intuitiveness of the result leads me to wonder if something was wrong with their analysis. The experiment seems ripe for this kind of statistical error.
Are the data and analytical processes used in this study available anywhere online?