In a 1974 speech Richard Feynman famously described an example of confirmation bias in science:

One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It's a little bit off because he had the incorrect value for the viscosity of air. It's interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan's, and the next one's a little bit bigger than that, and the next one's a little bit bigger than that, until finally they settle down to a number which is higher. Why didn't they discover the new number was higher right away? It's a thing that scientists are ashamed of—this history—because it's apparent that people did things like this: When they got a number that was too high above Millikan's, they thought something must be wrong—and they would look for and find a reason why something might be wrong. When they got a number close to Millikan's value they didn't look so hard. And so they eliminated the numbers that were too far off, and did other things like that...

I saw many publications reference this part of the speech, but to my surprise I have not been able to find the plot mentioned in the bolded sentence. I don't genuinely doubt Feynman's claims, but it seems fitting given the subject to ask for evidence rather than accept it blindly as an argument from authority!

Was anyone able to replicate this plot? How many data points could they find? Is the data as monotonic as Feynman claims, or did he omit some outliers for clarity?

  • 6
    Feynman is not implying the plot exists published somewhere. – Fizz May 23 at 17:23
  • 3
    There is already an SE question about this: hsm.stackexchange.com/questions/264/… – DJClayworth May 23 at 17:54
  • 1
    Looking at the numbers in the HSM thread, if anyone had confirmation bias, it looks like it might have been Feynman in this case. It seems like the next measurements are actually significantly larger, with measurements after that only showing slight changes. This seems more like what you would expect if the scientists had no real bias, and is purely the result of better experimental setup and analysis (i.e. having the right viscosity of air, or using a method that avoids that error). – JMac May 23 at 19:52
  • 4
    Having done the Millikan experiment once (50-odd years ago), I can believe that the new figures tracked as Feynman suggests. Much of the experimental result depends on the mathematical analysis done on the raw values, and it is easy to screw that up, or decide to discard values that don't "agree" with accepted values. I witnessed this in my class, where most students did the analysis wrong but got "better" values. – Daniel R Hicks May 23 at 21:41
  • 7
    This question shouldn't be closed. It's a notable enough parable in the sciences, so it seems like a good question for SE.Skeptics. If anyone finds the linked content a sufficient basis for an answer, then they can construct an answer based on it here. – Nat May 24 at 3:32

You must log in to answer this question.

Browse other questions tagged .