Who Are The Champions?

Having spent now over seven years applying statistical methods to the analysis and prediction of games of Australian Rules football I've become entirely comfortable - if I was feeling uncharitable I might even say blasé - about treating the outcome of these games as random variables with definable statistical characteristics.

The TAB Bookmaker, I've discovered to precisely no-one's surprise, does an inordinately good job of assessing teams' chances, as evidenced by the high level of what's known as "calibration" that his head-to-head forecasts exhibit. A well-calibrated forecaster is one for whom outcomes rated as 70% chances by him or her transpire about 70% of the time and, more generally, outcomes that he or she rates as X% chances occur about X% of the time. The TAB Bookmaker is exceptionally well-calibrated or, put another way and in the context of more than just head-to-head predictions, he's a very good estimator of the distributions of the random variables that describe different aspects of the outcomes in a football game.

But even he can't predict the outcome of every game or every aspect of a game, only estimate the relative likelihoods of potential outcomes, and if anyone has a financial interest in being able to foretell the future, it's a commercial bookmaker. Granted, it isn't especially profound of me to claim that the outcome of a sporting contest can't be known with certainty beforehand, even by someone with the data and motivation to do so if it were possible, but the extent to which the outcome appears to be determined "on the day" is particularly astonishing to me.

Consider the following statistics, based on the performance of the TAB Bookmaker, which I've observed and analysed over these seven years: 

• About 25-35% of contests are won by the "wrong" team, by which I mean that they're won by the team assessed by the Bookmaker as being less likely to win. 
• Statistically, only about 30-35% of the variability in the final victory margin of a game can be explained by the Bookmaker's pre-game assessment of the relative strengths of the competing teams.
• About half the time, the Bookmaker errs in his assessment of the final game margin by more than plus or minus 4 goals - an 8 goal range in total.

From a gambling point of view these observations are of no special import. Indeed, the Bookmaker's prices incorporate the observed variability since, if they did not, then wagering on, for example, favourites would be ridiculously lucrative. The TAB Bookmaker acknowledges the variability in outcomes and prices accordingly. This suggests that even he recognises that a huge proportion of the result of any single game is due to things that were unforeseeable before the contest began, events or talents that were only observed "on the day".

How much of this unpredictability is generally down to the superior skills of the team that eventually won and how much is, instead, due to essentially random factors - or, at least, factors outside the control of the two teams - is an interesting question to ponder.

As a theoretical exercise, consider the same game being replayed say a thousand times with each of the teams exhibiting identical skill levels in each game (whatever that means), so that the only cause of differences in the outcome, if any, are attributable to pure chance. Would all one thousand contests produce the same outcome? If not, how much variability would there be in the result across the thousand replicates?

The very fact that the TAB Bookmaker is able to allow for the variability in outcomes of real games in his pricing is, to me, tellng in itself. Surely if the causes of deviation from the expected result were specific to "on the day" but not truly random factors they'd not exhibit sufficient order to permit the Bookmaker to make such allowances in his prices. It's only because the random factors are indeterminate for a particular game but determinate, on average, across a sufficiently large number of games, that he can perform such a feat. That's indicative of results being "drawn" from a "distribution", surely.

In essence it comes down to whether or not you believe there is inherent, irreducible randomness in a sporting outcome that's not a consequence of the teams' relative skills on the day and, if there is, how much it contributes to the final outcome of a game and, moreover, to a season.

Sports commentators and, I'd argue, most sports fans, seem to premise their discussion of sporting results assuming that, a few bad umpiring decisions or blatantly obvious "bounces of the ball" aside, the outcome was somehow the "right" one, that the "better team won on the day" - in short that, in the normal course, randomness does not play dice with the footballing universe.

I'm not so sure.

Why I think it matters is because it goes to how we should treat our champion teams. By convincing ourselves that their success was, in the main, their own doing, we feel justified in celebrating their achievements as deserved reward for superior skill - and, as a consequence, in treating the team they defeated as less worthy if nonetheless valiant. But what if that's equivalent to congratulating the roulette player who picks "red" and doubles her money while deriding the "sucker" who chose "black" and lost it all? Or, to pick an example that's not so obviously determined by chance, of celebrating the victor in a game of backgammon while deriding the loser.

One of the best ways, of course, to reduce the element of chance in your assessment of a team's skill level is to view its performance across a series of contests. Based on that, there's clear merit in a 6-month long home-and-away season to winnow the 18 teams down to 8 finalists, but to then use a 4-week 9-game series to anoint an ultimate champion then seems a trifle odd.

Maybe the English Premier League know what they're doing ... but then we'd miss out on the drama of Grand Finals like the Swans-Hawks game of 2012.