Has the free market been taken too seriously or not seriously enough?

In a recent essay in the New York Times Magazine about why so many economists failed to predict the current crisis, Paul Krugman argues that a big piece is the cheerleading of Panglossian free marketeers, which helped to hide from view the possibility that markets can screw up, perhaps in very disastrous ways. Sure, there were a handful of skeptics and nay-sayers, but according to Krugman, they were too small in number or decibel level to get the attention of mainstream economists.

Not knowing much about the history of economic thought, I decided to see for myself by searching the "economics" section of JSTOR for articles that discuss markets failing. If Krugman's right, these should either be low in absolute number (whether they're up or down, they aren't enough to get people's attention), or if they're high in number, they should have been declining (as the cheerleading swept across the profession).

I've previously shown that the idea of money illusion declined in popularity from the mid-1970s onward, which is more of the kind of imperfections that Krugman is talking about. I could look for other ideas related to behavioral economics or behavioral finance, but these focus on how psychology affects economic behavior. I'm not sure they're the best measure of how common are ideas about the market failing. So, I looked instead at market failure ideas related to information -- these don't imply a wrinkle here or there due to some quirk of psychology, but to larger market failures. The most well known example is George Akerlof's hypothetical example of "the market for lemons" (used cars).

The phrases I searched for are "asymmetric information," "adverse selection," and "network externalities." I threw in "irrational exuberance" out of curiosity. If those terms are unfamiliar, I can't explain them at length, except to note that they are hypotheses about how markets can fail. (For instance, if the buyer knows something the seller doesn't, or vice versa; or if people value a technology not only for its use to them as individuals but also in relation to how big the network of users is.) They have Wikipedia entries, though.

I standardized the counts first by dividing by the total number of articles, which gives the % of all articles containing the keyword. This just tells us how prevalent the idea is, but it may be more useful to standardize by comparing the counts of the market failure ideas to the counts of articles that mention some completely basic idea in economics. That way, we can see how much respect the new ideas are given relative to time-tested ones. (It is like a person in 1997 comparing the stock prices of the dot-com companies to stocks in more mature industries.) I chose "supply and demand" as the fundamental concept to compare them to -- that's about as basic and popular of an economic concept as you can think of.

The pattern looks the same no matter what I use as the standardizing term, so I've stuck with the % of all articles measurement for all terms, although I've also included a graph with the comparison to "supply and demand" for two keywords to show just how much attention they have been given. Here are the results:



Clearly these ideas have only been gaining in popularity among economists. Indeed, the pioneers in this line of hypothesis-generating have received the Nobel Prize in economics for this work -- an event that should make us skeptical of Krugman's claim about how clueless economists have been, re: the possibility of markets screwing up. The broadest terms are currently pushing around 10% of all articles -- not too shabby. Moreover, since the late 1990s, the idea of "asymmetric information" has been on par with or even above the idea of "supply and demand" in how commonly discussed they are. It's hard to imagine beating "supply and demand" in a popularity contest, but there you have it.

(Indeed, perhaps it is the infatuation with such ideas that blinded many economists to basic concepts like supply and demand -- for instance, that housing prices were shooting up because demand was shooting up, as it does during a bubble, not because of some radical shift in the interest rate, construction costs, or population size, as Robert Shiller showed years before the bubble burst.)

Krugman did not claim that if only economists had paid more attention to market failure theories involving asymmetric information, they would've seen the crisis coming. But his broader claim about how sanguine the profession had become about the superiority of markets and the small chance that they'd screw up big -- this claim is wrong. Asymmetric information economists and behavioral economists have recently won the Nobel Prize, and we've just seen that the former ideas have only gained in popularity, rivaling even the respect given to the concept of "supply and demand"!

I think the poor showing of economists is not due to theoretical blinders so much as it is to their desire to fit in with polite society (which in turn may influence what theoretical hunches they have). No one wants to rain on the parade during a boom or period of apparent great moderation. Others will take offense, turn them out, and so on. The two economists who had the most foresight, Robert Shiller and Nouriel Roubini, don't care how well accepted they are. Shiller's mind is that of an engineering nerd, which doesn't register a social faux pas very sensitively. And Roubini is someone who gets a thrill out of telling other people how stupid or clueless they are (this is the main appeal for academics in many fields). So obviously he won't care if he's an outcast.

Many view economists as asocial or autistic nerds, but that doesn't seem to be so true -- they have fully functional and thirsty social antennae, but they were the ones left out of the cool kids crowd in high school, and who desperately want to be part of respectable society as adults. Hence all their name-dropping about Mozart, Picasso, exotic cuisine, and which neighborhoods in Zagreb are the most authentic. "See, we're not just a bunch of math geeks! Let us join the party!"

Clearly seeing the current crisis would have required them to call polite society a bunch of idiots -- for believing that home prices can only go up, up, up -- or recklessly ideological for using PC to debauch lending standards. Not the best way to get invited to the right dinner parties, the board of directors at the right institutions, or the conferences of right-thinking society.