Empirical Bias and Sigma for Selected Probability Predictors

In the previous blog I revisited the five parameter simulation model for the contest between Bookmaker and Punter and came up with sets of parameters - or 'scenarios' as I called them - that were associated with, for example, better expected profit outcomes.

To estimate the true Home team probabilities for each of the games from 2007 to 2012, for that blog I fitted a simple binary logit model and then used its outputs to calculate an empirical Bias and Sigma for the TAB Bookmaker's LPSO-based Implicit Home team probabilities.

This blog applies that same approach to the TAB Bookmaker's implicit Home team probabilities if we assume he sets prices to equalise risk or to equalise overround on the two teams. It also applies the approach to the probability predictions of ProPred, WinPred and Head-to-Head (Unadjusted). The results for the latter Predictor are particularly interesting because this Predictor is what's used for the Head-to-Head Fund.

The top half of the table provides, on the left-hand side, the estimated Bias for each Predictor for various subsets of the rounds in a season. It shows, for example, that: 

• The TAB Bookmaker, assuming we can infer his Home team probability estimates using the results of the LPSO approach (which means that his estimate of the Home team probability is the inverse of the Home team's price minus 1.0281%), tends to underestimate the Home team's chances in the early parts of the season and in the Finals, and overestimate them otherwise.
• If, instead, we assume that the TAB Bookmaker is Overround-Equalising or Risk-Equalising then he consistently underestimates the true chances of the Home team, especially in the early parts of the season and in the Finals.
• The pattern of bias for the Head-to-Head Probability Predictor across the season mirrors that of an LPSO-like TAB Bookmaker but tends to a slightly greater bias towards the Home team. 

If we look instead at the bias on a season-by-season basis we find that: 

• An LPSO-like TAB Bookmaker is close to unbiased in every season
• An Overround-Equalising or Risk-Equalising TAB Bookmaker underestimates Home teams' chances, though less so in more-recent seasons
• The Head-to-Head Probability Predictor has demonstrated a bias towards Home teams in every season but 2007. In most seasons, 2011 the notable exception, the bias has been quite small however. 

The right-hand side of the table provides estimates of sigma and reveals that: 

• The TAB Bookmaker based Predictors all have smaller estimated sigmas than any of the MAFL Probability Predictors shown, but the TAB Bookmaker Predictors have shown increasing sigmas over the past five seasons - good news indeed in light of the results in the previous blog, which demonstrated how vitally the ROI to Kelly-Staking depended on a less-precise Bookmaker.
• Sigmas for the Head-to-Head Probability Predictor have generally been in the 10.5% to 12.5% range and have been decreasing over the past three seasons, which is also good news from an expected profit viewpoint. (It's interesting to note that the last time an algorithm similar to Head-to-Head made a significant profit for MAFL was in 2009. Having said that, this was also the year in which the TAB Bookmaker sigma estimates were smallest too.) 

In the context of the results that I reported in the previous blog, I think the key empirical results from this blog are that the TAB Bookmaker can be thought of as a Bookmaker with zero bias and a 5-5.5% sigma, and the Head-to-Head Probability Predictor can be thought of as a Punter with a +1-2% bias and a 10-12% sigma.