Tag Archives for statistics

World Crisis Index

Intrade, the Ireland-based prediction market, has launched a World Crisis Index. This index is a sum of the prices of 8 current markets Intrade is making in the area of global crisis, including a markets on recessions and growth rates in industrialized countries, US unemployment rates, the possibility of new US military action, and other issues. This sum is then normalized and reported. The Intrade markets first came to my attention via an email from Robin Hanson, who is arguably the world’s leading expert in prediction markets. Intrade had a good deal of success in predicting the outcomes of the last election cycle.

I followed the market fluctuations in the electoral issues pretty closely last year, specifically through Intrade’s partnership with Rasmussen Reports. What was interesting to me was how well the markets predicted changes in press coverage, from positive to negative or more interestingly, from sparse to dense and vice versa.

Dennett and modern positivism

Philosopher Daniel Dennett contributed an essay to the John Brockman edited collection What Are You Optimistic About? about the role that modern information technology might have on the growth of rationality and consequently, an increase in secularism and atheism. The essay, like most of the essays that Brockman solicits, is thought-provoking. Alas, the first thought it provoked for me was something along the lines of, “Clearly, Dennett has no faith in humanity’s ability to be stupid with greater speed and efficiency than before!”

The existence of technology will no more prevent religious fundamentalists of either the Muslim or Christian stripe from acting irrationally, anymore than the technology of the first Industrial Revolution caused an expansion of Enlightenment rationality as the 19th century positivists believed.

Google Spreadsheets and the Long Tail

I’ve been using Google Spreadsheets more recently to all the little one-pager list type spreadsheets that I make. When I noticed that they had added a notifications system, so that you can get an email when a collaborator updates a spreadsheet, or even a particular cell in the sheet, it sparked an idea. What if a cell could contain a short script, or a URL that points to a script on another server, that returned a number. The sheet could the update dynamically based on the result of the script.

This could be integrated with AdWords or Google Analytics, to return data from there that could feed custom analysis spreadsheets or could point to an internal database to use proprietary business data. I could also see data providers, e.g. the National Weather Service or Digg, providing statistics feeds that folks could slice and dice for their own purposes.

In a sense, this could be part of the long tail of the “super-cruncher”” phenomenon. Its hard to get good data to play with right now. A standard platform, or at least a standard access protocol, for raw numbers would open up opportunities to crowdsource data analysis.

Review and commentary on “Super Crunchers” by Ian Ayres

I recently read Ian Ayres’ excellent book, Super Crunchers. For folks who read and enjoyed Freakonomics, this book is a must-read, covering more cases where clever statistical analyses have uncovered interesting and useful results. The goal in writing the book, according to Ayres, was to encourage people to learn to think statistically. On the other side of the link is a discussion of some errors in experimental design, why their treatment in Ayres’ book frustrates me and why the average person should care.

Continue reading →