« Horrid stuff 2 | Main | Mirror, mirror »



This reminds me of expert interviews using the Delphi design (invented by the RAND cooperation).
"Experts" are anonymously asked to give their opinion towards a contingent and vague subject. In a second wave they asked the same questions again including some feed-back (average) from the first round. This should enhance the results of the study because real experts stay with their opinion while the others change their minds and follow the feed-back (average).

Chris Hibbert

The studies on Prediction Markets have shown them to produce better forecasts than any other mechanism they've been tested against. There seems to be an effect that people who know more bet more, and people who are relatively ill-informed learn that they are losing money and drop out. The fact that people are backing their opinions with money means that the presence of "bad information" attracts those with better information; they can make money by betting against those who are less informed.

James Surowiecki's "The Wisdom of Crowds" is a good reference. You can find a lot more at midasoracle.com, or by looking up Robin Hanson or Justin Wolfers.

Disclaimer: I'm writing Open Source Prediction Market Software called Zocalo.


Chris, I'll visit the websites and educate myself some more on the research out there. What is mystifying to me is what the well-informed experts might gain by diluting their predictions by averaging them with ill-formed ones.

Chris Hibbert

Experts seem to overestimate their abilities. Philip Tetlock has been studying expert's abilities; he wrote Expert Political Judgment, and gave a recent talk at the Long Now Foundation. I blogged about the talk at Overcoming Bias. Tetlock says that Prediction Markets can do better than all but the very best individual experts. If you're planning to rely on experts, the evidence seems to show that unless you've fortuitously chosen one of the top few percent, you'd do better with a market. Markets are more reliable.

The comments to this entry are closed.

Kaiser Fung. Business analytics and data visualization expert. Author and Speaker.
Visit my website. Follow my Twitter. See my articles at Daily Beast, 538, HBR.

See my Youtube and Flickr.

Book Blog

Link to junkcharts

Graphics design by Amanda Lee

The Read

Keep in Touch

follow me on Twitter