Diffusion and Advection

Recent blog posts by Michael Schaffer and Hendrik Herzberg nicely characterise the feeling of futility provoked by trying to track and understand all of the variables involved in the ongoing US election. It is very difficult to get a sense both of political events, and of the media and popular reactions to those events. Still, perhaps predictions are possible without understanding. Indeed, there is very good reason to think that the best available predictions in a case of this kind will go independently of understanding (see here and here for why). Unfortunately, the depression associated with attempting to understand politics is not lightened by examining the statistical factors behind political judgement instead. To wit: Alexander Todorov, Anesu N. Mandisodza, Amir Goren, and Crystal C. Hall, "Inferences of Competence from Faces Predict Election Outcomes", in Science, Vol. 308, No. 5878, 10 June 2005, pp. 1623–1626.

We show that inferences of competence based solely on facial appearance predicted the outcomes of U.S. congressional elections better than chance (e.g., 68.8% of the Senate races in 2004) and also were linearly related to the margin of victory. These inferences were specific to competence and occurred within a 1-second exposure to the faces of the candidates. The findings suggest that rapid, unreflective trait inferences can contribute to voting choices, which are widely assumed to be based primarily on rational and deliberative considerations.

Now of course, good polling methods will do better than facial appearance judgements with respect to predicting outcomes, and do not produce any depressing thoughts, unless we include the depressing thought that these polls tend to be close even in elections such as this one where the superior candidate is clear. (Still, I would be interested in seeing a test of facial appearance judgements on Obama and McCain). Even better than polling, however, are prediction markets: Joyce E. Berg, Forrest D. Nelson and Thomas A. Rietz, "Prediction market accuracy in the long run", in International Journal of Forecasting, Vol. 24, No. 2, April-June 2008, pp. 285–300.

“Prediction markets” are designed specifically to forecast events such as elections. Though election prediction markets have been being conducted for almost twenty years, to date nearly all of the evidence on efficiency compares election eve forecasts with final pre-election polls and actual outcomes. Here, we present evidence that prediction markets outperform polls for longer horizons. We gather national polls for the 1988 through 2004 U.S. Presidential elections and ask whether either the poll or a contemporaneous Iowa Electronic Markets vote-share market prediction is closer to the eventual outcome for the two-major-party vote split. We compare market predictions to 964 polls over the five Presidential elections since 1988. The market is closer to the eventual outcome 74% of the time. Further, the market significantly outperforms the polls in every election when forecasting more than 100 days in advance.

This is from a special issue of the journal on election forecasting, in which the introduction (published online on 25 March 2008) to the issue concludes:

To summarize, of the forecasters here who brave a clear prediction, all name the Democrats. Of course, these are mostly speculations about the forecasts that will be firmed up by later in the summer and early fall. Even so, they suggest that this may be a good year for the Democrats. But when, as forecasters, we climb out on the limb, we know it may get sawed off.

For times when the drum of expert judgements becomes deafening, the graph for the Iowa Electronic Markets 2008 US Presidential Election Winner Takes All Market can be seen here. Free markets might not be good for economies, but—in contrast to the human brain—they are surprisingly good at synthesising information.