Intrade 2004, 2008, & 2012 U.S. Presidential election forecasts

Today is the 13^th anniversary of Intrade’s collapse.

Before Kalshi, Polymarket, and even PredictIt, Intrade was the dominant prediction market, drawing the most traders and cited by news media.

Several weeks ago, in another pointless and uninformed argument on social media, someone referenced Intrade prices as being bad at forecasting events. That made me wonder how accurate it was at forecasting, particularly the U.S. Presidential elections.

I found several interesting looks at Intrade election markets, including:

In Forecasting Elections: Comparing Prediction Markets, Polls, and Their Biases, David Rothschild compares Intrade prices for the 2008 U.S. Presidential election with 538 and his own model of polls. Rothschild includes a “debiased” version of Intrade prices that corrects for the longshot bias common in prediction markets. His analysis found “in the 2008 election cycle, FiveThirtyEight’s debiased poll-based forecasts were, on average, slightly more accurate than Intrade’s raw prediction market-based prices. But when prediction markets are properly debiased, they are more accurate and contain more information than debiased polls; this advantage is most significant for forecasts made early in the cycle and in not-certain races (i.e., the races typically of most interest).”
Gwern Branwen’s analysis of the 2012 US Presidential election forecasts comparing Intrade, 538, and several election forecasters. He found “the best overall performance seems to be by Drew Linzer and Wang & Holbrook, while Nate Silver appears as somewhat overrated and the famous Intrade prediction market turned in a disappointing overall performance.”

Thanks to Professor Panos Ipeirotis at NYU, we have much of the Intrade trading data from 2003 to 2013. You can find his Intrade data archive here.

Below, we’re going to look at the turbulent history of Intrade and then measure accuracy using state-level forecasts made in the three presidential elections with data available.

A Brief History of Intrade

Founded in 2000 by professional traders Ron Bernstein and brothers John & Sean McNamara, the sports-focused platform launched on August 2^nd of the following year, growing to a user base of roughly 500 traders by the end of the year. Because US gambling laws would be unfavorable to the company, they made the decision to incorporate in Dublin. Nick Berry built the exchange platform, and accountant John Delaney was brought on board as vice president of finance.

Intrade would run into the same issue as many prediction market startups today: trading binary options is not intuitive to the casual sports bettor.

So Bernstein made a couple of key decisions: He created a new company, called Tradesports, and made Delaney the CEO. Intrade would remain separate and lease its technology to Tradesports. Bernstein held on to his stake in Intrade, but returned to the U.S. and went back to floor trading. (Later the two companies would be recombined.) It was John Delaney’s business to run.
The mysterious life and death of Intrade’s CEO

Delaney was now in charge, and more reorganizations would come:

In 2004, Tradesports was reorganized into Trade Exchange Network Limited, which operated Intrade.com, Tradesports.com, WallStreetSports.com, and Prediction-X.com.

In 2007, that entity was further split into three corporate bodies: Intrade the Prediction Market Limited, Tradesports Ltd, and Prediction Exchange Services Limited. This essentially transferred non-sports prediction markets and technology-related intellectual property to Intrade; however, Intrade and Tradesports would operate out of the same offices and share bank accounts.

By 2006, Intrade had grown to 60,000 traders, and with growth came attention from regulators.

The CFTC issued a consent order and “announced the filing and simultaneous settlement of charges” against the Trade Exchange Network in October 2005 for violating US law by “soliciting and accepting orders from U.S. residents for commodity options not otherwise excepted or exempted from the Commission’s ban on options.”

Both MF Global and the Minneapolis Grain Exchange were interested in purchasing Intrade, which could have brought it back to the United States, but John Delaney refused to sell the exchange despite its financial problems.

Delaney would tragically die climbing Mount Everest in 2011. In November 2012, a few weeks after the US Presidential election, the CFTC would go in for the kill, filing new charges of allowing US citizens to trade and violating the order from 2005.

And things continued to get even worse for Intrade. An auditor’s report from February 2013 found money missing, millions paid to John Delaney in 2010 and 2011, and auditors couldn’t even find all the records:

We have not obtained all the information and explanations we consider necessary for the purpose of our audit. In our opinion proper books of accounts were not kept by the company during the period being audited.
Intrade The Prediction Market Limited Abridged accounts for the year ended 31 December 2011

Intrade would freeze trader accounts and announce its closure on March 10, 2013.

US Presidential elections of 2004, 2008, & 2012

We’re going to look at InTrade performance in the U.S. Presidential elections of 2004 (Bush v Kerry), 2008 (Obama v McCain), and 2012 (Obama v Romney) by measuring the daily state-level Brier scores based on each GOP candidate’s closing price that day.

The Brier score is essentially the mean squared error for each day of market prices by state. Each market error is the difference between the actual outcome and the market’s probability for that day. That error is then squared, which has the consequence of eliminating negative values and penalizing extreme predictions. The Brier score is the simple mean of the squared errors for all the forecasts. For example, given a share price of $0.80/share and a resolution NO:

(Probability – Resolution)² = (0.8-0)² = (0.8)² = 0.64

We then average the squared error for all 50 states + Washington, D.C. for that day. Lower is better for Brier scores, with a score of 0 being perfect accuracy and 1 being perfectly inaccurate.

Below are daily Brier scores from April 1 through the day before Election Day in all three Presidential elections:

Note that the 2012 data only uses 48 states because of a lack of pricing data for New York, Maryland, and Delaware.

Intrade market forecasts generated generally low Brier scores before Election Day, and a clear downward trend over time shows the markets continually improving accuracy. The lone exception would be the 11 days (Sept 4-14, 2008) after the Republican National Convention. The McCain campaign was finally gaining traction with a post-convention bounce putting him ahead of Obama in the polls, only for Lehman Brothers to collapse on September 15, the same day John McCain publicly declared the “fundamentals of the economy are strong.”

The three Brier scores themselves mean relatively little given a lack of comparison with other forecasts. The Rothschild article linked above compares 2008 Intrade prices with FiveThirtyEight state-level model forecasts and found FiveThirtyEight outperformed (i.e., lower error) than raw Intrade data, but correcting Intrade prices for the longshot bias generated more accurate forecasts than FiveThirtyEight. It would be interesting to track down the 2012 FiveThirtyEight forecasts and see if that holds true in another election.

Here is a summary of the Intrade market data available in Professor Ipeirotis’ archive:

2004 U.S. Presidential election

Election Day: November 2, 2004
Date range: 2003-05-30 to 2004-11-07 (528 calendar days)
No closing price available on the following nine dates: 4/24/2004, 3/24/2004, 3/19/2004, 1/1/2004, 12/25/2003, 12/26/2003, 11/18/2003, 10/7/2003, & 9/19/2003.
Only price and trading data for markets on Bush’s chances of winning.

2008 U.S. Presidential election

Election Day: November 4, 2008
Date range: 2006-11-12 to 2008-11-19 (739 calendar days)
Price and trade data for both Republican and Democratic candidates.

2012 U.S. Presidential election

Election Day: November 6, 2012
Date range: 2011-01-24 -> 2012-11-10 (657 calendar days)
DE & MD data had no activity while NY had none until Election Day.
Price and trade data for both Republican and Democratic candidates, and third party candidates in certain states.