John Cassidy’s 2009 book, “How Markets Fail,” drives the final nail in the coffin of the Efficient Market Hypothesis.

Well, perhaps the penultimate nail – as I’ll explain later. It is the most compelling argument I have read that we need a new and improved theory of markets, a theory that subsumes the efficient market hypothesis, much as Einstein’s relativity theory subsumed Newtonian physics.

As a bonus, Cassidy’s book is an excellent layman’s introduction to economics. It begins by tracing the history of economic thought from Adam Smith through Friedrich Hayek to Arrow-Debreu, culminating in the perfection of the idea that the collective result of individuals pursuing their self-interest is that social welfare is maximized – what Cassidy calls “Utopian Economics.”

An historic interchange

The book begins with an historic exchange that deserves reproduction here at length. It took place on October 23, 2008, in the chambers of the U.S. House of Representatives, between Henry Waxman, chairman of the House Committee on Oversight and Government Reform, and Alan Greenspan, former chairman of the Federal Reserve. Cassidy describes its climax thus:

Waxman returned to his notes and started reading again. “‘I do have an ideology. My judgment is that free, competitive markets are by far the unrivaled way to organize economies. We have tried regulations. None meaningfully worked.’” “That was your quote,” [Waxman] said. “You had the authority to prevent irresponsible lending practices that led to the subprime mortgage crisis. You were advised to do so by many others. Now our whole economy is paying the price. Do you feel that your ideology pushed you to make decisions that you wish you had not made?”

[Greenspan responded:] “To exist, you need an ideology. The question is whether it is accurate or not. What I am saying to you is, yes, I found a flaw. I don’t know how significant or permanent it is, but I have been very distressed by that fact.”

… but [Waxman] wasn’t finished. “In other words, you found that your view of the world, your ideology, was not right,” he said. “It was not working?”

“Precisely,” Greenspan replied. “That’s precisely the reason I was shocked. Because I had been going for 40 years, or more, with very considerable evidence that it was working exceptionally well.”

This is the stuff of which Shakespearean tragedies are made. Greenspan was the ultimate empiricist. He was thus, in the end, one of the very few to respond publicly to the challenge of his intellectual rival John Maynard Keynes: “When the facts change, I change my mind – what do you do, sir?”

It is easy for people to forget that we are now seeing the obvious flaws in “utopian economics” only in retrospect. Until the financial crisis of 2007-2008, the vast majority of economists and political thinkers were converts to what is now being referred to – pejoratively – as “fundamentalist” free-market theory. It took a new and unprecedented experience for many to change their minds. Having finally seen the truth from which we were blinded, we should remember an essential part of Greenspan’s words: “I found a flaw. I don’t know how significant or permanent it is…”