Google's commitment to data-driven decisions is well reported, and the company has been ridiculed for the "50 shades of blue" episode, when then Google executive Marissa Meyer led a project testing the impact of using different coloured links in ads.
But a new insight proves that the company significantly benefitted from the experiment, to the tune of $200m.
I'm not skeptical so much about whether the tests actually happened, or even if Google projected that the change would generate that much extra revenue, but the counter-intuitiveness of the result leads me to wonder if something was wrong with their analysis. The experiment seems ripe for this kind of statistical error.
Are the data and analytical processes used in this study available anywhere online?