On a Los Angeles street corner in 2000, I was the “inside man” in a classic con game called the pigeon drop. A magician named Dan Harlan orchestrated it for a television series I cohosted called Exploring the Unknown (type “Shermer, con games” into Google). Our pigeon was a man from whom I asked directions to the local hospital while Dan (the “outside man”) moved in and appeared to find a wallet full of cash on the ground. After it was established that the wallet belonged to neither of us and appeared to have about $3,000 in it, Dan announced that we should split the money three ways.

I objected on moral grounds, insisting that we ask around first, which Dan agreed to do only after I put the cash in an envelope and secretly switched it for an envelope with magazine pages stuffed in it. Before he left on his moral crusade, however, Dan insisted that we each give him some collateral (“How do I know you two won’t just take off with the money while I’m gone?”). I enthusiastically offered $50 and suggested that the pigeon do the same. He hesitated, so I handed him the sealed envelope full of what he believed was the cash (but was actually magazine pages), which he then tucked safely into his pocket as he willingly handed over to Dan his entire wallet, credit cards and ID. A few minutes after Dan left, I acted agitated and took off in search of him, leaving the pigeon standing on the street corner with a phony envelope and no wallet!

After admitting my anxiety about performing the con (I didn’t believe I could pull it off) and confessing a little thrill at having scored the goods, I asked Dan to explain why such scams work. “We are that way as the human animal,” he reflected. “We have a conscience, but we also want to go for the kill.” Indeed, even after we told our pigeon that he had been set up, he still believed he had the three grand in his pocket!

Greed and the belief that the payoff is real also led high-rolling investors to fuel Wall Street financier Bernard Madoff’s record-breaking $50-billion Ponzi scheme in which he kept the money and paid an 8 to 14 percent annual annuity with cash from new investors. As long as more money comes in than goes out, such scams can continue, which this one did until the 2008 market meltdown, when more investors wanted out than wanted in. But there were other factors at work as well, as explained by the University of Colorado at Boulder psychiatry professor Stephen Greenspan in his new book The Annals of Gullibility (Praeger, 2008), which, with supreme irony, he wrote before he lost more than half his retirement investments in Madoff’s company! “The basic mechanism explaining the success of Ponzi schemes is the tendency of humans to model their actions, especially when dealing with matters they don’t fully understand, on the behavior of other humans,” Greenspan notes.

The effect is particularly powerful within an ethnic or religious community, as in 1920, when the eponymous Charles Ponzi promised a 40 percent return on his fellow immigrant Italian investors’ money through the buying and selling of postal reply coupons (the profit was supposedly in the exchange rate differences between countries). Similarly, Madoff targeted fellow wealthy Jewish investors and philanthropists, and that insider’s trust was reinforced by the reliable payout of moderate dividends (so as not to attract attention) to his selective client list, to the point that Greenspan said he would have felt foolish had he not grabbed the investment opportunity.

The evolutionary arms race between deception and deception detection has left us with a legacy of looking for signals to trust or distrust others. The system works reasonably well in simple social situations with many opportunities for interaction, such as those of our hunter-gatherer ancestors. But in the modern world of distance, anonymity and especially complicated investment tools (such as hedge funds) that not one in a thousand really understands, detecting deceptive signals is no easy feat. So as Dan reminded me, “If it sounds too good to be true, it is.”

Here’s a supreme irony for you. About six months ago a colleague of mine named Stephen Greenspan, a psychiatry professor at the University of Colorado, sent me a book manuscript to review and blurb for him (a blurb is one of those back jacket endorsements from someone who hopefully knows something about the subject of the book). Greenspan’s book is called Annals of Gullibility (Praeger, 2009, due out in January), and it includes chapters on gullibility in literature and folktales (Pinocchio, Gulliver), in religion (end-of-the-world predictions), in war and politics (the Trojan Horse), in criminal justice (child witnesses), in science (cold fusion), and in finance (Ponzi schemes). It’s a great read and an excellent reference source that, as I wrote in my blurb, “belongs on the bookshelves of skeptics and scientists, not to mention politicians and policy analysts, especially before they go to war.”

Well, last week Stephen emailed me a query letter about writing an article for Skeptic on Ponzi schemes, based on the chapter in his book, and — here’s the irony — it would recount how he lost a huge chunk of his retirement investments (to the tune of hundreds of thousands of dollars) not to the market collapse (like the rest of us) but by investing in none other than Bernard Madoff’s now-infamous Ponzi scheme. Yup, a psychiatrist who wrote the book on gullibility got taken.

What I am skeptical about here, however, is not Madoff and his scam, but the media’s portrayal of his investors as suckers for falling for it. My question is this: how was anyone outside of the Security and Exchange Commission (SEC) to know? What signs and signals were there for the average investor to see? Madoff was head of NASDAQ for three years and his investment company apparently consistently returned annual dividends to his investors in the range of 8% to 14% — healthy but not outrageous (apparently his golf scores were similarly rigged to make him appear good but not great, shooting 80–89 every round). One could make the case that the SEC should have known (indeed, they were warned in 2005 that Madoff was running a Ponzi scheme), or that investment experts who monitor the business should have been suspicious (and some of them were but their voices went unheard). But how would a college professor in Colorado, or Joe the Plumber in Puckerbrush, Pennsylvania, or you and me as Joe Sixpack investors know that Bernie Madoff was a latter-day Charles Ponzi?

Madoff’s deal was especially effective because it was what is called an affinity scam, where you appeal to those in your social group, in this case Jewish investors. It makes you feel like you’re an insider, a member of an exclusive club, and as such you would surely not be scammed by one of your own. If, say, you were working in the entertainment industry and Stephen Spielberg or Jeffery Katzenberg (both clients of Madoff) phoned to tell you about this sound investment opportunity that was by invitation only and that they could get you in for a minimum of $100,000, and that they had been invested for years in this program and had reliably received annual dividend checks ranging from 8% to 14% on their money, what would you do? You’d most likely jump at the opportunity. In fact, in his article in the forthcoming issue of Skeptic (and in this week’s eSkeptic), Stephen Greenspan recalls that he felt like he would have been a fool not to capitalize on this opportunity. Was he a fool for so doing? Only in hindsight. But what foresight was there?

By way of analogy, in 2000 I co-hosted a television series for the Fox Family Channel called Exploring the Unknown, in which we produced a segment on cons and scams, one of which was the infamous three-card monte. We employed the help of a professional magician named Dan Harlan who set up a cardboard box at a street mall in Santa Monica to show us how easy it is to sucker people into giving him their money (watch the segment on YouTube). It’s a relatively simple game. There are three cards on top of the box. One of them is, say, the Queen of Spades (the money card), while the other two are numbered cards. Your task is to follow the Queen as the magician/conman rapidly moves the cards around the boxtop. When he is finished with half a dozen moves, you place your bill ($5, $10, or $20) in front of the card that you think is the Queen. If you are right he matches your bill. If you are wrong he takes your bill. Unless you are a shill working for the magician/conman (who is signaled through an agree-upon sign where the Queen is located so that he can occasionally win), you will always lose. Why? Because the three-card monte involves one sleight-of-hand move whereby when he is rapidly moving the cards about and occasionally showing you the Queen on the bottom of two cards in his hand, he moves his ring (or pinky) finger down a card from holding the top card to holding the Queen, so that when he appears to be tossing the bottom card (Queen) onto the box top, he is actually tossing down the top card and thereby moving the queen to a different spot. You can’t see it. For the show we had to ask Dan to make the move in slow motion for us, and even then we had to slow down the tape in order to see the move.

Would you be fool enough to fall for the three-card monte? Most of us would say yes, because we have heard that it’s a con, or we’ve seen shows like the one I produced, or we’ve read about it in a magazine or a book. But if you had never heard of the game and saw one being played, by the information of your senses you would see some people winning (you wouldn’t know that these are shills) and you would not see the sleight-of-hand move. So it would not be unreasonable to believe that you stand a reasonable chance of winning. In other words, what signs would there be that a con was underway?

The analogy holds for Madoff-level investment scams. Short of just being skeptical of all investment technologies (which, obviously, most of us are not and that’s what helps fuel the modern economy), how are any of us to know which investment companies are legit and which are not? The SEC? We’ve seen how well that works, so what’s an investor to do?

My answer is … diversify. What’s your answer?