Surprising new research shows that crowds are often smarter than individuals
In 2002 I served as the “phone a friend” for the popular television series Who Wants to Be a Millionaire. When my acquaintance was stumped by a question, however, he elected to “poll the audience” instead. His choice was wise not only because I did not know the answer but because the data show that the audience is right 91 percent of the time, compared with only 65 percent for “experts.”
Although this difference may in part be because the audience is usually queried for easier questions, something deeper is at work here. For solving a surprisingly large and varied number of problems, crowds are smarter than individuals. This is contrary to what the 19th-century Scottish journalist Charles Mackay concluded in his 1841 book, Extraordinary Popular Delusions and the Madness of Crowds, a staple of skeptical literature: “Men, it has been well said, think in herds. It will be seen that they go mad in herds, while they only recover their senses slowly, and one by one.” This has been the dogma ever since, supported by sociologists such as Gustave Le Bon, in his classic work The Crowd: A Study of the Popular Mind: “In crowds it is stupidity and not mother wit that is accumulated.”
Au contraire, Monsieur Le Bon. There is now overwhelming evidence, artfully accumulated and articulated by New Yorker columnist James Surowiecki in his enthralling 2004 book, The Wisdom of Crowds (Doubleday), that “the many are smarter than the few.” In one experiment, participants were asked to estimate the number of jelly beans in a jar. The group average was 871, only 2.5 percent off the actual figure of 850. Only one of the 56 subjects was closer. The reason is that in a group, individual errors on either side of the true figure cancel each other out.
A similar result was discovered in an example so counterintuitive that it startles. When the U.S. submarine Scorpion disappeared in May 1968, a naval scientist named John Craven assembled a diverse group of submarine experts, mathematicians and salvage divers. Instead of putting them in a room to consult one another, he had each of them give a best guesstimate — based on the sub’s last known speed and position (and nothing else) — of the cause of its demise and its rate and steepness of descent, among other variables. Craven then computed a group average employing Bayes’s theorem, a statistical method wherein a probability is assigned to each component of a problem. The Scorpion’s location on the ocean floor was only 220 yards from the averaged prediction.
Stranger still was the stock market’s reaction on January 28, 1986, the day the space shuttle Challenger exploded. Of the four major shuttle contractors — Lockheed, Rockwell International, Martin Marietta and Morton Thiokol — the last (the builder of the defective solidrocket booster) was hit hardest, with a 12 percent loss, compared with only 3 percent for the others.
A detailed study of the market (a sizable crowd, indeed!) by economists Michael T. Maloney of Clemson University and J. Harold Mulherin of Claremont McKenna College could find no evidence of insider trading or media focus on the rocket booster or on Morton Thiokol. Given four possibilities, the masses voted correctly.
Not all crowds are wise, of course — lynch mobs come to mind. And “herding” can be a problem when the members of a group think uniformly in the wrong direction. The stock market erred after the space shuttle Columbia disaster on February 1, 2003, for example, dumping stock in the booster’s manufacturer even though the boosters were not involved.
For a group to be smart, it should be autonomous, decentralized and cognitively diverse, which the committee that rejected the foam-impact theory of the space shuttle Columbia while it was still in flight was not. In comparison, Google is brilliant because it uses an algorithm that ranks Web pages by the number of links to them, with those links themselves valued by the number of links to their page of origin. This system works because the Internet is the largest autonomous, decentralized and diverse crowd in history, IMHO.