The Calibration of the Head-to-Head Fund Algorithm

In the previous blog we considered the logarithmic probability score on ProPred, WinPred and the TAB bookie and found that the TAB bookie was the best calibrated of the three and that relative tipping performance was somewhat unrelated to relative probability scores.

For the Head-to-Head Fund, whose job in life is to make money, the key question is to what extent do its probability scores relative to the TAB bookie's shed light on its money-making prowess.

Here's the comparison:

The 'Adjusted' notation in this table is to reflect the fact that the probability scores are based on the adjusted probabilities from the Head-to-Head (H2H) algorithm. These adjustments are twofold: 

1. If the original assessed probability is more than 25% higher than the probability implicit in the bookie's prices then the H2H probability is ratcheted back to be exactly 25% higher 
2. If the Home team price is greater than $5 then the H2H probability is set to the probability implicit in the bookie's prices 

Rightmost in this table is a column recording the return that the H2H Fund would have made in that year were it full-Kelly wagering. It seems that it's possible for the H2H Fund to make money even when its probability score is significantly worse than the TAB bookies.

But this misses one key point: H2H doesn't wager on every game, only those in which it thinks the TAB bookie has underestimated the Home team's chances. So the relevant comparison is only for those games in which the H2H Fund wagered.

So now it seems that it is possible to make money despite having a poorer probability score than the bookie's, but perhaps only slightly poorer. To investigate this a little further, let's repeat the analysis y season segment.

This analysis suggests a few general principles: 

• It's possible to make money wagering even in those season segments where the Fund's probability score is slightly lower than the bookie's
• Losses accrue, however, when the difference is too large
• Profits will accrue almost certainly in those season segments where the Fund's probability score exceeds the bookie's 

These principles, coupled with a further analysis, provide context for the decision to prevent the H2H Fund from wagering on Home teams priced over $5. Here's a table showing how the unadjusted H2H Fund algorithm performs for various Home team price ranges:

On the left are the probability scores for all games. There you can see how poorly the unadjusted algorithm does relative to the bookie for Home team prices over $5. When we restrict our attention to just those games in which the unadjusted H2H Fund would have wagered, the relative performance for Home teams priced over $5 becomes even worse.

Lastly, let's look at the performance of the adjusted H2H Fund across the various Home team price ranges.

Once again the principles I described earlier hold up and we can see that the H2H Fund makes the majority of its profit on very short-priced and on longshot, but not outrageously so, Home team favourites.

During the upcoming season it'll be interesting to see if we can identify the specific characteristics of the games in which the H2H Fund algorithm tends to produce high probability scores, as these are also likely to be the characteristics of the games in which it can be most profitable.