The following letter was sent to me in response to my column in Scientific American (which generated hundreds of letters in response, so I penned the following response) in which I discussed the now-infamous (and infuriatingly counter-intuitive) probability problem called the Monty Hall Problem, or the Three Door Problem, in which a contestant chooses one of three doors, behind one of which is a car and the other two goats. Monty then reveals what’s behind one of the other doors (only ever showing a goat and never showing you your own door pick), which is always a goat, then asks if you want to change doors. Most people say it doesn’t matter because now it’s 50/50, but the correct answer is that you should always switch, which will give you a two-thirds chance of winning. There are simulations of the game online, but my correspondent took it upon himself to test the game with his own computer program. Here are his very interesting results, which also nicely show the scientific method at work:

Mr. Shermer,

I am writing to thank you for your articles in Scientific American, specifically the one in the October 2008 issue discussing the ‘Monty Hall Problem’. Thanks to your essay, I think I finally understand the scientific method.

After reading about the ‘Monty Hall Problem’, I couldn’t shake the idea that switching doors shouldn’t make a difference. I knew that I must be wrong, but couldn’t get my head around the problem; I couldn’t get to sleep for a couple of hours that night either. So, instead of just believing that I was right or wrong and leaving it at that, I decided to see if I could find any objective data that would support one view or the other.

I wrote a little Visual Basic application within an Access database and ran 100,000 sessions where the contestant switched doors every time. The contestant was successful a little over 62% of the time. This seemed to lean to the conclusion that switching leads to a two-thirds success rate, but 62.2% seemed odd. I ran 1,000,000 sessions to see if the numbers be more definitive; they weren’t, still 62.2%. So, I looked through the database tables where I recorded the results to see what was going on. It was then that the true meaning of the scientific method became apparent to me. Looking through the data, I developed a new theory of the ‘Monty Hall Problem’ and why the strategy of switching doors should be successful two-thirds of the time. The new theory was elegant, the logic seemed clear, even obvious, and it seemed to agree with the data. The remaining problem was what was happening with the missing four-and-a-half percent. My suspicion was that this was caused by the random number generator I was using to pick the door with the car behind it and the door the contestant chose during each trial not being random enough. I rewrote the function choosing these doors, attempting to make them more random and ran 100,000 new trials and ended up with a success rate of 66.43%, close enough to satisfy me that the switching strategy is indeed the way to go.

As I mentioned, this little exercise opened my eyes to the true meaning and power of the scientific method. I was confronted with two competing and mutually exclusive theories explaining how something works. Instead of stubbornly standing by my own gut feeling, or believing another theory simply on faith, I ran an experiment to see if either theory would be supported or disproved. Examining the data led me to support the switching strategy and to develop a new theory explaining why this is so. I also developed a new theory to explain the remaining discrepancies in the data, ran a second, refined experiment, and gained further support for the theory behind the switching strategy.

I’ve read most of Stephen Jay Gould and Carl Sagan, I even subscribe to Scientific American. I always thought that I believed in the scientific method. However, it took your article, and its inspiring me to use the scientific method for myself to finally truly understand it.

Thank you,
Douglas Millar

Thank you Douglas Millar!
Michael Shermer

According to 55 percent of 350,000 people from 70 countries who participated online in Richard Wiseman’s Laugh Lab experiment (discussed in last month’s column), this is the world’s funniest joke:

Two hunters are out in the woods when one of them collapses. He doesn’t seem to be breathing, and his eyes are glazed. The other guy whips out his phone and calls the emergency services. He gasps, “My friend is dead! What can I do?” The operator says, “Calm down. I can help. First, let’s make sure he’s dead.” There is a silence, then a shot is heard. Back on the phone, the guy says, “Okay, now what?”

So say the data, but according to Wiseman’s personal narrative describing how the research was actually conducted (in his new book Quirkology), he believes that “we uncovered the world’s blandest joke — the gag that makes everyone smile but very few laugh out loud. But as with so many quests, the journey was far more important than the destination. Along the way we looked at what makes us laugh, how laughter can make you live longer, how humor should unite different nations, and we discovered the world’s funniest comedy animal.” Chickens notwithstanding, such first-person accounts in popular science books that include the journey and not just the destination afford readers a glimpse into how science is really carried out.

Formal science writing — what I call the “narrative of explanation” — presents a neat and tidy step-by-step process of Introduction- Methods-Results-Discussion, grounded in a nonexistent “scientific method” of Observation- Hypothesis-Prediction- Experiment followed in a linear fashion. This type of science writing is like autobiography, and as the comedian Stephen Wright said, “I’m writing an unauthorized autobiography.” Any other kind is fiction. Formal science writing is like Whiggish history — the conclusion draws the explanation toward it, forcing facts and events to fall neatly into a causal chain where the final outcome is an inevitable result of a logical and inevitable sequence.

Informal science writing — what I call the “narrative of practice” — presents the actual course of science as it is interwoven with periodic insights and subjective intuitions, random guesses and fortuitous findings. Science, like life, is messy and haphazard, full of quirky contingencies, unexpected bifurcations, serendipitous discoveries, unanticipated encounters and unpredictable outcomes. This chaotic process helps to explain, in part, the phenomenal success in recent decades of first-person popular accounts by scientists of how they actually did their research. The effect is especially noteworthy in works exploring the peculiarities of life.

Steven Levitt’s and Stephen Dubner’s Freakonomics (William Morrow, 2006) illuminates the power of incentives through certain oddities. For instance, that most drug dealers live with their mothers because only the top guys make the big bucks while the rest bide their time and pay their dues or that baby names tell us about the motives of parents. Cornell University professor Robert Frank’s The Economic Naturalist: In Search of Explanations for Everyday Enigmas (Basic, 2007) employs the principle of costbenefit analysis to explain such idiosyncrasies as why drive-up ATM keypads have Braille dots (because it is cheaper to make the same machine for both drive-up and walk-up locations), why brown eggs are more expensive than white eggs (because there is less demand and the hens that lay them are larger and consume more food), why it is harder to find a taxi in the rain (because more people use them when it is raining, most cabbies reach their fare goals earlier in the day), and why milk is stored in rectangular cartons but soft drinks come in round cans (because it is handier to drink soda directly from a round can but easier to pour and store milk in a rectangular carton).

In my October column I railed against the artificial (and odious) ranking of technical science writing over popular science writing. I suggested that the latter should be elevated to a more exalted standing of “integrative science,” where good science writing integrates data, theory and narrative into a useful and compelling work. Here I add that exploring the minutiae of life, especially on the quirky borderlands of science, makes the scientific process more accessible to everyone. Where a narrative of explanation might read something like “the data lead me to conclude…,” a narrative of practice reads more like “Huh, that’s weird…”

Weirdness trumps data in the biography of science.