« Horrid stuff 2 | Main | Mirror, mirror »



This reminds me of expert interviews using the Delphi design (invented by the RAND cooperation).
"Experts" are anonymously asked to give their opinion towards a contingent and vague subject. In a second wave they asked the same questions again including some feed-back (average) from the first round. This should enhance the results of the study because real experts stay with their opinion while the others change their minds and follow the feed-back (average).

Chris Hibbert

The studies on Prediction Markets have shown them to produce better forecasts than any other mechanism they've been tested against. There seems to be an effect that people who know more bet more, and people who are relatively ill-informed learn that they are losing money and drop out. The fact that people are backing their opinions with money means that the presence of "bad information" attracts those with better information; they can make money by betting against those who are less informed.

James Surowiecki's "The Wisdom of Crowds" is a good reference. You can find a lot more at midasoracle.com, or by looking up Robin Hanson or Justin Wolfers.

Disclaimer: I'm writing Open Source Prediction Market Software called Zocalo.


Chris, I'll visit the websites and educate myself some more on the research out there. What is mystifying to me is what the well-informed experts might gain by diluting their predictions by averaging them with ill-formed ones.

Chris Hibbert

Experts seem to overestimate their abilities. Philip Tetlock has been studying expert's abilities; he wrote Expert Political Judgment, and gave a recent talk at the Long Now Foundation. I blogged about the talk at Overcoming Bias. Tetlock says that Prediction Markets can do better than all but the very best individual experts. If you're planning to rely on experts, the evidence seems to show that unless you've fortuitously chosen one of the top few percent, you'd do better with a market. Markets are more reliable.

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.


Post a comment

Your Information

(Name is required. Email address will not be displayed with the comment.)


Link to Principal Analytics Prep

See our curriculum, instructors. Apply.
Marketing analytics and data visualization expert. Author and Speaker. Currently at Columbia. See my full bio.

Book Blog

Link to junkcharts

Graphics design by Amanda Lee

The Read

Good Books

Keep in Touch

follow me on Twitter