The following letter was sent to me in response to my column in Scientific American (which generated hundreds of letters in response, so I penned the following response) in which I discussed the now-infamous (and infuriatingly counter-intuitive) probability problem called the Monty Hall Problem, or the Three Door Problem, in which a contestant chooses one of three doors, behind one of which is a car and the other two goats. Monty then reveals what’s behind one of the other doors (only ever showing a goat and never showing you your own door pick), which is always a goat, then asks if you want to change doors. Most people say it doesn’t matter because now it’s 50/50, but the correct answer is that you should always switch, which will give you a two-thirds chance of winning. There are simulations of the game online, but my correspondent took it upon himself to test the game with his own computer program. Here are his very interesting results, which also nicely show the scientific method at work:

Mr. Shermer,

I am writing to thank you for your articles in Scientific American, specifically the one in the October 2008 issue discussing the ‘Monty Hall Problem’. Thanks to your essay, I think I finally understand the scientific method.

After reading about the ‘Monty Hall Problem’, I couldn’t shake the idea that switching doors shouldn’t make a difference. I knew that I must be wrong, but couldn’t get my head around the problem; I couldn’t get to sleep for a couple of hours that night either. So, instead of just believing that I was right or wrong and leaving it at that, I decided to see if I could find any objective data that would support one view or the other.

I wrote a little Visual Basic application within an Access database and ran 100,000 sessions where the contestant switched doors every time. The contestant was successful a little over 62% of the time. This seemed to lean to the conclusion that switching leads to a two-thirds success rate, but 62.2% seemed odd. I ran 1,000,000 sessions to see if the numbers be more definitive; they weren’t, still 62.2%. So, I looked through the database tables where I recorded the results to see what was going on. It was then that the true meaning of the scientific method became apparent to me. Looking through the data, I developed a new theory of the ‘Monty Hall Problem’ and why the strategy of switching doors should be successful two-thirds of the time. The new theory was elegant, the logic seemed clear, even obvious, and it seemed to agree with the data. The remaining problem was what was happening with the missing four-and-a-half percent. My suspicion was that this was caused by the random number generator I was using to pick the door with the car behind it and the door the contestant chose during each trial not being random enough. I rewrote the function choosing these doors, attempting to make them more random and ran 100,000 new trials and ended up with a success rate of 66.43%, close enough to satisfy me that the switching strategy is indeed the way to go.

As I mentioned, this little exercise opened my eyes to the true meaning and power of the scientific method. I was confronted with two competing and mutually exclusive theories explaining how something works. Instead of stubbornly standing by my own gut feeling, or believing another theory simply on faith, I ran an experiment to see if either theory would be supported or disproved. Examining the data led me to support the switching strategy and to develop a new theory explaining why this is so. I also developed a new theory to explain the remaining discrepancies in the data, ran a second, refined experiment, and gained further support for the theory behind the switching strategy.

I’ve read most of Stephen Jay Gould and Carl Sagan, I even subscribe to Scientific American. I always thought that I believed in the scientific method. However, it took your article, and its inspiring me to use the scientific method for myself to finally truly understand it.

Thank you,
Douglas Millar

Thank you Douglas Millar!
Michael Shermer

Imagine that you are a contestant on the classic television game show Let’s Make a Deal. Behind one of three doors is a brand-new automobile. Behind the other two are goats. You choose door number one. Host Monty Hall, who knows what is behind all three doors, shows you that a goat is behind number two, then inquires: Would you like to keep the door you chose or switch? Our folk numeracy — our natural tendency to think anecdotally and to focus on small-number runs — tells us that it is 50–50, so it doesn’t matter, right?

Wrong. You had a one in three chance to start, but now that Monty has shown you one of the losing doors, you have a twothirds chance of winning by switching. Here is why. There are three possible three-doors configurations: (1) good, bad, bad; (2) bad, good, bad; (3) bad, bad, good. In (1) you lose by switching, but in (2) and (3) you can win by switching. If your folk numeracy is still overriding your rational brain, let’s say that there are 10 doors: you choose door number one, and Monty shows you door numbers two through nine, all goats. Now do you switch? Of course, because your chances of winning increase from one in 10 to nine in 10. This type of counterintuitive problem drives people to innumeracy, including mathematicians and statisticians, who famously upbraided Marilyn vos Savant when she first presented this puzzle in her Parade magazine column in 1990.

The “Monty Hall Problem” is just one of many probability puzzles that physicist Leonard Mlodinow of the California Institute of Technology presents in his delightfully entertaining new book The Drunkard’s Walk (Pantheon, 2008). His title employs the metaphor (sometimes called the “random walk”) to draw an analogy between “the paths molecules follow as they fly through space, incessantly bumping, and being bumped by, their sister molecules,” and “our lives, our paths from college to career, from single life to family life, from first hole of golf to eighteenth.” Although countless random collisions tend to cancel one another out because of the law of large numbers — where improbable events will probably happen given enough time and opportunity — every once in a great while, “when pure luck occasionally leads to a lopsided preponderance of hits from some particular direction … a noticeable jiggle occurs.” We notice the improbable directional jiggle but ignore the zillions of meaningless and counteracting collisions.

In the Middle Land of our ancient evolutionary environment, which I introduced in Part 1 of this column last month, our brains never evolved a probability network, and thus our folk intuitions are ill equipped to deal with many aspects of the modern world. Although our intuitions can be useful in dealing with other people and social relationships (which evolved as common and important for a social primate species such as ours when we were struggling to survive in the harsh environs of the Paleolithic), they are misleading when it comes to such probabilistic problems as gambling. Let’s say you are playing the roulette wheel and you hit five reds in a row. Should you stay with red because you are on a “hot streak,” or should you switch because black is “due”? It doesn’t matter, because the roulette wheel has no memory, yet gamblers notoriously employ both the “hot streak fallacy” and the “dueness fallacy,” much to the delight of casino owners.

Additional random processes and our folk numeracy about them abound. The “law of small numbers,” for example, causes Hollywood studio executives to fire successful producers after a short run of box-office bombs, only to discover that the subsequent films under production during the producer’s reign became blockbusters after the firing. Athletes who appear on Sports Illustrated’s cover typically experience career downturns, not because of a jinx but because of the “regression to the mean,” where the exemplary performance that landed them on the cover is itself a low probability event that is difficult to repeat.

Extraordinary events do not always require extraordinary causes. Given enough time, they can happen by chance. Knowing this, Mlodinow says, “we can improve our skill at decision making and tame some of the biases that lead us to make poor judgments and poor choices … and we can learn to judge decisions by the spectrum of potential outcomes they might have produced rather than by the particular result that actually occurred.” Embrace the random. Find the pattern. Know the difference.