Nov 19, 2015
Nov 19, 2015

The Brier Score method

With West Ham, on one end of the scale, and Liverpool and Chelsea on the other, this article measures the extent of the surprise factor for each EPL team by applying a technique used to evaluate the accuracy of weather predictions.

The current Premiership table leads to a couple of unexpected outcomes – Leicester in third position, West Ham in top six and Chelsea in a part of the table where you have to scroll down.

One of the best available predictors for an outcome, especially if you believe in efficient markets, are betting odds. Hence, teams that perform differently than what predicted are either over-performers or under-performers.

To measure the extent to which each team deviates from what the odds imply, we use a Brier Score per team. The Brier Score, as originally introduced in 1950, is a measure of the effectiveness of weather predictions.

Calculating the Brier Score

As probabilities implied by betting markets add up to more than 100%, we first adjust the odds pro-rata to determine the probabilities of each outcome for a match.

The Brier Score per match is the sum of the square difference of the probability and actual results. Let’s take the match Liverpool vs Crystal Palace on 8th November. Pinnacle odds implied a 58.3% chance of Liverpool to win and a 24.5% chance of draw. The outcome was a Crystal Palace win at a predicted possibility of just 17.1%.

The probabilities for Win:Draw:Loss were 0.583:0.245:0.171 with the actual result being 0:0:1, where one stands for the actual outcome. The differences are 0.583:0.245:0.829. The sum of the square differences are 0.5832 + 0.2452 + 0.8292 = 1.0875. This is the Brier Score for this match.

The Brier score can range from 0, if odds guessed the outcome directly by stating there is 100% chance of only one outcome to be correct, to 2, if odds implied that one outcome is certain and that did not occur.

However, if we had to just throw outcomes at random and say that a home win, draw or away win are equally likely, then our Brier score would be 0.667, irrespective of the outcome.

The table below shows the Brier score per team in the Premiership. What we note is that some of the surprises are actually vindicated in the stats: Chelsea, Liverpool and West Ham are not performing as expected while for Arsenal and Manchester City it is business as usual.

Football is unpredictable. That unpredictability gives the sport its appeal inspires the fans and also gives betting punters a headache week in, week out. Using a verification measure like the Brier Score gives betters a more systematic point of view as to what is happening in the league right now.

However, should it be the only measure used, it will still be limited by the beautiful fact that football is a sport played by, organized by, coached by and watched by human beings. As such, always price in the possibility of a process error.

Teams with high Brier Score

Who would have ever thought that mighty Chelsea from last year would implode months after winning the league rather easily? One can never fully predict such a thing even though this is exactly what we are supposedly most familiar with: the human factor.

It is the human factor that gels a bunch of random human beings into a well-oiled machine. And this is what Chelsea seems to have lost this year, if the rumours about a mutiny going on at Chelsea Football Club are to be believed. On the flipside, who could have predicted that Leicester and West Ham would have such brilliant runs?

One could argue that almost every year an averagely sized team packs up a far greater punch than what is expected of them. New manager Slaven Bilic probably was a deciding factor in this no-nonsense away conquering West Ham side. Such a thing is easy to say in hindsight but what the clever pundit strives to do is see things in foresight and this is once again where the human factor comes in.

Observing West Ham, Bilic has brought in a continental approach to defending. Whereas the defending modus operandi of the smaller Premier League clubs is generally to park the bus, Bilic leaves the bus stationary but running, rather than parked. His teams defend deeply but attack rapidly and take their chances well. Incidentally, Claudio Ranieri is doing the exact same thing with Leicester.

Can Klopp turn the tide?

Will Liverpool's appointing of a new manager contribute to a safer club to bet on? Up to this point Liverpool's Brier score seems to suggest otherwise. Post-Klopp appointment, their average Brier score is 0.887 higher than the pre-Klopp appointment. This is due to the least likely outcome occurring in three of their four premiership matches to date (loss vs. Crystal Palace at home, draw versus Tottenham and win versus Chelsea).

His first home match versus Southampton ended in a draw even if it was only about 1/5 likely. An expensively assembled team who year in year out (apart from that Luis Suarez agonisingly quasi-league winning year) seem to under-perform.

As it stands Liverpool is a team capable of winning against anyone, on a good day, and equally able to lose against anyone, on a bad one. It remains to be seen whether Klopp can make good days more frequent, thus making Liverpool safer to bet on.