13

I was torn between posting this on Cross Validated, User Experience, or here, but here goes:

Many have heard about how Google, not wanting to rely on the hunches of designers, tested 41 shades of blue for their links and found that the best performing shade led to $US200M in revenue a year.

Google's commitment to data-driven decisions is well reported, and the company has been ridiculed for the "50 shades of blue" episode, when then Google executive Marissa Meyer led a project testing the impact of using different coloured links in ads.

But a new insight proves that the company significantly benefitted from the experiment, to the tune of $200m.

I'm not skeptical so much about whether the tests actually happened, or even if Google projected that the change would generate that much extra revenue, but the counter-intuitiveness of the result leads me to wonder if something was wrong with their analysis. The experiment seems ripe for this kind of statistical error.

Are the data and analytical processes used in this study available anywhere online?

Improve this question
8
  • 2
    So what is your question, exactly? Did Google do 41 tests? Did they achieve a sufficient degree of statistical accuracy with those tests? Did those tests lead to $200m increased revenue?
    – Flimzy
    Dec 23, 2015 at 13:45
  • Neither "is this analysis wrong?" nor "Is this study available online?" are on topic at this site.
    – DJClayworth
    Dec 23, 2015 at 16:46
  • 2
    @DJClayworth Some questions on this site are skeptical that a study took place. Isn't it on topic to be skeptical of the study itself?
    – panofsteel
    Dec 23, 2015 at 23:17
  • 1
    @Flimzy "Did those tests lead to $200m increased revenue" gets to the heart of my question quite well.
    – panofsteel
    Dec 23, 2015 at 23:23
  • 2
    As far as I can tell this isn't a testable claim, since it's based in Google's actual CPM as reported by one of their UK directors, which is anecdotal evidence which we can't easily repeat.
    – Avery
    Dec 24, 2015 at 0:32

0

Reset to default

Browse other questions tagged .