The following letter was sent to me in response to my column in Scientific American (which generated hundreds of letters in response, so I penned the following response) in which I discussed the now-infamous (and infuriatingly counter-intuitive) probability problem called the Monty Hall Problem, or the Three Door Problem, in which a contestant chooses one of three doors, behind one of which is a car and the other two goats. Monty then reveals what’s behind one of the other doors (only ever showing a goat and never showing you your own door pick), which is always a goat, then asks if you want to change doors. Most people say it doesn’t matter because now it’s 50/50, but the correct answer is that you should always switch, which will give you a two-thirds chance of winning. There are simulations of the game online, but my correspondent took it upon himself to test the game with his own computer program. Here are his very interesting results, which also nicely show the scientific method at work:

Mr. Shermer,

I am writing to thank you for your articles in Scientific American, specifically the one in the October 2008 issue discussing the ‘Monty Hall Problem’. Thanks to your essay, I think I finally understand the scientific method.

After reading about the ‘Monty Hall Problem’, I couldn’t shake the idea that switching doors shouldn’t make a difference. I knew that I must be wrong, but couldn’t get my head around the problem; I couldn’t get to sleep for a couple of hours that night either. So, instead of just believing that I was right or wrong and leaving it at that, I decided to see if I could find any objective data that would support one view or the other.

I wrote a little Visual Basic application within an Access database and ran 100,000 sessions where the contestant switched doors every time. The contestant was successful a little over 62% of the time. This seemed to lean to the conclusion that switching leads to a two-thirds success rate, but 62.2% seemed odd. I ran 1,000,000 sessions to see if the numbers be more definitive; they weren’t, still 62.2%. So, I looked through the database tables where I recorded the results to see what was going on. It was then that the true meaning of the scientific method became apparent to me. Looking through the data, I developed a new theory of the ‘Monty Hall Problem’ and why the strategy of switching doors should be successful two-thirds of the time. The new theory was elegant, the logic seemed clear, even obvious, and it seemed to agree with the data. The remaining problem was what was happening with the missing four-and-a-half percent. My suspicion was that this was caused by the random number generator I was using to pick the door with the car behind it and the door the contestant chose during each trial not being random enough. I rewrote the function choosing these doors, attempting to make them more random and ran 100,000 new trials and ended up with a success rate of 66.43%, close enough to satisfy me that the switching strategy is indeed the way to go.

As I mentioned, this little exercise opened my eyes to the true meaning and power of the scientific method. I was confronted with two competing and mutually exclusive theories explaining how something works. Instead of stubbornly standing by my own gut feeling, or believing another theory simply on faith, I ran an experiment to see if either theory would be supported or disproved. Examining the data led me to support the switching strategy and to develop a new theory explaining why this is so. I also developed a new theory to explain the remaining discrepancies in the data, ran a second, refined experiment, and gained further support for the theory behind the switching strategy.

I’ve read most of Stephen Jay Gould and Carl Sagan, I even subscribe to Scientific American. I always thought that I believed in the scientific method. However, it took your article, and its inspiring me to use the scientific method for myself to finally truly understand it.

Thank you,
Douglas Millar

Thank you Douglas Millar!
Michael Shermer

Because I am often introduced as a “professional skeptic,” people feel compelled to challenge me with stories about highly improbable events. The implication is that if I cannot offer a satisfactory natural explanation for that particular event, the general principle of supernaturalism is preserved. A common story is the one about having a dream or thought about the death of a friend or relative and then receiving a phone call five minutes later about the unexpected death of that very person.

I cannot always explain such specific incidents, but a principle of probability called the Law of Large Numbers shows that an event with a low probability of occurrence in a small number of trials has a high probability of occurrence in a large number of trials. Events with million-to-one odds happen 295 times a day in America.

In their delightful book Debunked! (Johns Hopkins University Press, 2004), CERN physicist Georges Charpak and University of Nice physicist Henri Broch show how the application of probability theory to such events is enlightening. In the case of death premonitions, suppose that you know of 10 people a year who die and that you think about each of those people once a year. One year contains 105,120 five-minute intervals during which you might think about each of the 10 people, a probability of one out of 10,512 — certainly an improbable event. Yet there are 295 million Americans. Assume, for the sake of our calculation, that they think like you. That makes 1/10,512 × 295,000,000 = 28,063 people a year, or 77 people a day for whom this improbable premonition becomes probable. With the well-known cognitive phenomenon of confirmation bias firmly in force (where we notice the hits and ignore the misses in support of our favorite beliefs), if just a couple of these people recount their miraculous tales in a public forum (next on Oprah!), the paranormal seems vindicated. In fact, they are merely demonstrating the laws of probability writ large.

Another form of this principle was suggested by physicist Freeman Dyson of the Institute for Advanced Study in Princeton, N.J. In a review of Debunked! (New York Review of Books, March 25), he invoked “Littlewood’s Law of Miracles” (John Littlewood was a University of Cambridge mathematician): “In the course of any normal person’s life, miracles happen at a rate of roughly one per month.” Dyson explains that “during the time that we are awake and actively engaged in living our lives, roughly for eight hours each day, we see and hear things happening at a rate of about one per second. So the total number of events that happen to us is about thirty thousand per day, or about a million per month. With few exceptions, these events are not miracles because they are insignificant. The chance of a miracle is about one per million events. Therefore we should expect about one miracle to happen, on the average, every month.”

Despite this cogent explanation, Dyson concludes with a “tenable” hypothesis that “paranormal phenomena may really exist,” because, he says, “I am not a reductionist.” Further, Dyson attests, “that paranormal phenomena are real but lie outside the limits of science is supported by a great mass of evidence.” That evidence is entirely anecdotal, he admits. But because his grandmother was a faith healer and his cousin was a former editor of the Journal for Psychical Research and because anecdotes gathered by the Society for Psychical Research and other organizations suggest that under certain conditions (for example, stress) some people sometimes exhibit paranormal powers (unless experimental controls are employed, at which point the powers disappear), Dyson finds it “plausible that a world of mental phenomena should exist, too fluid and evanescent to be grasped with the cumbersome tools of science.” Freeman Dyson is one of the great minds of our time, and I admire him immensely. But even genius of this magnitude cannot override the cognitive biases that favor anecdotal thinking.

The only way to find out if anecdotes represent real phenomena is controlled tests. Either people can read other people’s minds (or ESP cards), or they can’t. Science has unequivocally demonstrated that they can’t — QED. And being a holist instead of a reductionist, being related to psychics, or reading about weird things that befall people does not change this fact.

In his 1916 poem “A Coat,” William Butler Yeats rhymed: “I made my song a coat/Covered with embroideries/Out of old mythologies/From heel to throat.”

Read “religion” for “song,” and “science” for “coat,” and we have a close approximation of the deepest flaw in the science and religion movement, as revealed in Yeats’s denouement: “But the fools caught it,/Wore it in the world’s eyes/As though they’d wrought it./ Song, let them take it / For there’s more enterprise/In walking naked.”

Naked faith is what religious enterprise was always about, until science became the preeminent system of natural verisimilitude, tempting the faithful to employ its wares in the practice of preternatural belief. Although most efforts in this genre offer little more than scientistic cant and religious blather, a few require a response from the magisterium of science, if for no other reason than to protect that of religion; if faith is tethered to science, what happens when the science changes? One of the most innovative works in this genre is The Probability of God (Crown Forum, 2003), by Stephen D. Unwin, a risk management consultant in Ohio, whose early physics work on quantum gravity showed him that the universe is probabilistic and whose later research in risk analysis led him to this ultimate computation.

Unwin rejects most scientific attempts to prove the divine — such as the anthropic principle and intelligent design — concluding that this “is not the sort of evidence that points in either direction, for or against.” Instead he employs Bayesian probabilities, a statistical method devised by 18th-century Presbyterian minister and mathematician Reverend Thomas Bayes. Unwin begins with a 50 percent probability that God exists (because 50–50 represents “maximum ignorance”), then applies a modified Bayesian theorem:

The probability of God’s existence after the evidence is considered is a function of the probability before times D (“Divine Indicator Scale”): 10 indicates the evidence is 10 times as likely to be produced if God exists, 2 is two times as likely if God exists, 1 is neutral, 0.5 is moderately more likely if God does not exist, and 0.1 is much more likely if God does not exist. Unwin offers the following figures for six lines of evidence: recognition of goodness (D = 10), existence of moral evil (D = 0.5), existence of natural evil (D = 0.1), intranatural miracles (prayers) (D = 2), extranatural miracles (resurrection) (D = 1), and religious experiences (D = 2).

Plugging these figures into the above formula (in sequence, where the P_after figure for the first computation is used for the P_before figure in the second computation, and so on for all six Ds), Unwin concludes: “The probability that God exists is 67%.” Remarkably, he then confesses: “This number has a subjective element since it reflects my assessment of the evidence. It isn’t as if we have calculated the value of pi for the first time.”

Indeed, based on my own theory of the evolutionary origins of morality and the sociocultural foundation of religious beliefs and faith, I would begin (as Unwin does) with a 50 percent probability of God’s existence and plug in these figures: recognition of goodness (D = 0.5), existence of moral evil (D = 0.1), existence of natural evil (D = 0.1), intranatural miracles (D = 1), extranatural miracles (D = 0.5), and religious experiences (D = 0.1). I estimate the probability that God exists is 0.02, or 2 percent.

Regardless, the subjective component in the formula relegates its use to an entertaining exercise in thinking — on par with mathematical puzzles — but little more. In my opinion, the question of God’s existence is a scientifically insoluble one. Thus, all such scientistic theologies are compelling only to those who already believe. Religious faith depends on a host of social, psychological and emotional factors that have little or nothing to do with probabilities, evidence and logic. This is faith’s inescapable weakness. It is also, undeniably, its greatest power.