I would like to introduce a framework for assessing the impact of “Honest Mistakes”.
These are game changing decisions that officials get wrong because they are, errr, human (!). For example, wrong decisions on goals scored / disallowed, players sent off / not sent off.
The need for such a framework is, hopefully, obvious.
If you have ever tried to debate a contentious decision online or indeed in person (remember that?) you will know that the following is self-evident – the vast majority of supporters (including me):
- are at best partially blinded with unbreakable bias;
- will not change their opinion irrespective of the evidence presented; and
- don’t know the Laws of the Game.
So, my goal is to come up with a method of assessing Honest Mistakes and their impact.
Assessing Honest Mistakes
I will admit to being a Celtic supporter (obviously) and therefore have the same confirmation biases nearly all football supporters do in a partisan sport.
Furthermore, I have never refereed nor studied the Laws of the Game. It doesn’t stop me voicing an opinion on decisions, of course.
However, with tens of millions of pounds at stake through Champions League qualification, is that really good enough? Of course not, and in such circumstances my instinct is to ask an expert.
Therefore, I’d like to introduce one – the Yorkshire Whistler.
The Yorkshire Whistler is someone I have never personally met, but whom a mutual contact in the South Yorkshire football network (I live in Sheffield) put me in touch with. I am delighted and grateful he has agreed to adjudicate on contentious SPFL decisions.
Firstly, the need for anonymity is, sadly, self-evident, given the level of vitriol some resort to when debating the Scottish game.
The Yorkshire Whistler has 10 years refereeing experience after playing at semi-professional level. He is a Sheffield Wednesday supporter (an Owl AND a referee – poor guy!) with no more than a passing interest in Scottish football. In his own words:
“My knowledge and following of the game north of the border is basic and somewhat limited.
Other than watching the classified results on a Saturday afternoon to see if the ‘big 2’ have been upset, my only real interest is when the Celtic/Rangers derby matches come around.
I always take the opportunity to watch this fixture as a complete neutral and it’s a game that rarely disappoints!”
The Yorkshire Whistler will review incidents of note in the SPFL and provide me unbiased and expert assessment in the style of Ref Watch on Sky.
I will relay the verdicts, unvarnished, to you.
Impact of Sendings Off
The second pillar in my framework is again provided by experts. The Royal Statistical Society, no less.
The Red Card Cliche is an article that sets out to debunk the “it’s harder to play against 10 men” football myth. Which is up there with “you can only defend corners man to man” and “you have to play two up front to be successful” in the Football Myths League.
I digress.
This analysis assessed 1520 English Premier League matches to understand the impact of sendings off using regression techniques. I encourage you to read it but I will summarise.
Sendings off reduce the number of points a team is expected to win. The average impact is summarised thus:
| Team Suffering the Sending Off | Average Drop in Expected Points |
|
Home |
0.86 |
| Away |
0.35 |
I will cover the concept of Expected Points more fully below but for now accept that the outcome of every game is 3 points for win and 1 point for a draw and 0 for a loss. The values expressed above indicate the average number of points further away from 3 each team is following the sending off.
The reason a home team has more than twice the negative outcome from a sending off compared to the away team is due to the fact the home team has more starting expected points to lose.
I will use these averages to assess the impact of a sending off wrongly awarded or not awarded on expected points outcomes. They are averages meaning that the impact of a sending off in the 10th minute will differ from that in the 70th minute. But I don’t have the data to say what that impact is by game time or state. So, imperfect as it is, we’ll take the raw average.
Bear in mind the article notes “It seems safe to assume that if a weaker team (as measured by league points/ranking) has a player sent off, this is likely to have a bigger impact than when the stronger team loses a man.”.
In the context of the SPFL where Celtic and The Rangers are usually significantly stronger than the opposition, this is relevant.
That is, if a player is sent off (or not sent off) the impact is far greater when those clubs are involved.
However, lacking specific data on this, I will use the (understated) EPL averages above.
Expected Points
You may have come across the concept of Expected Points (xPts) to describe the league points teams are expected to have achieved based on Expected Goals For and Against. Unfortunately, this is different but has the same name!
Expected Points in this context is predicated on, again, you accepting the premise every match is played for 3,1,0 points as the outcome even if a cup tie.
Once you accept that, what the xPts model does is place a value on every goal scored (or disallowed) in terms of how much it adds to the points you can expected to win from the game up to 3.
It is best to use some specific examples:
- Souttar’s header for Hearts against Celtic was in the 89th minute of the game with the score 1-1. With only 1 minute plus added time to play, the xPts added to Hearts for that goal is 1.7. That is, with the scores level their xPts was already slightly more than 1 (as the home team) and that goal took it to over 2.7 xPts out of the maximum of 3 points for a win. Intuitively we all get an 89th minute goal allowing a team to lead dramatically changes the likely outcome with so few minutes left to equalise.
- Abada’s 16th minute opener against Dundee United recently added 0.7 to Celtics xPts. The goal was early in the game but home teams don’t often lose games from being in front. With 74 minutes left to play, it took Celtic from slightly above 1 xPts to just over 1.7 on the quest for 3. Dundee United’s equaliser 2 minutes later added 0.6 xPts to them.
- On 21st August Turnbull complete his hattrick in the home match versus St Mirren. Whilst this was undoubtedly an important personal landmark for Turnbull, it did virtually nothing to change the game. The goal was scored in 84th minute with Celtic already leading 4-0 and playing against 10 players (see above). Thus, the goal added 0 to Celtic’s xPts.
What this concept allows us to do is assess the impact of a goal wrongly disallowed or wrongly allowed in terms of the number of xPts it added / should have added had the correct decision been given.
For example, if Dundee had a goal disallowed against Celtic at home in the 10th minute and this decision was judged incorrect, assuming the score at the time was 0-0, Celtic “benefited” by 0.7 xPts from that decision.
We can then cumulatively tally the xPts outcomes that have occurred due to Honest Mistakes to see potentially what impact the correct decisions made on teams’ points totals.
Summary
We now have a framework to assess Honest Mistakes and their impact:
- using unbiased expert opinion – the Yorkshire Whistler
- impact of sendings off – Royal Statistics Society
- impact of goals on xPts – American Soccer Analysis
Importantly, this will not be MY opinion.
What I will do is present the decisions and impacts of the major contentious moments in SPFL. Tomorrow I will present the decisions from July and August in the SPFL and then another piece covering the rest of the season to date.
I will then look to update this as and when, maybe weekly the way things are going.
The officials and the corrupt sfa.spfl are doing a great job giving newco sevco their first ever title now theirs 40 million up for grabs at the end of this season and i bet any money the corrupt refs will be doing everything to make sure it goes over to the biggest cheats in scottish football oldco and newco just watch and as for all the clubs in the spfl are they scared to open mouths because i know my club is p.lawwell and the rest should get out of paradise all cowards everyone of them.
Capitalisation and punctuation really help others understand your point.
Great stuff Alan!
I’m almost too excited to read this properly, as I have been wondering about getting a framework for assessing this very point.
A couple of points have always stuck me on this (as a guy who makes a living from “sciency” stuff).
1) How to ensure setting the framework up is genuinely free from bias (as a Celtic fan and pondered this frequently I tend to focus only on items Celtic suffer from, and hence thereby inherently bias any ideas I’ve had. eg a system for weighting up value-of-free-kicks-given rather than pure number, so well done on the framework! ?
2) Particular to the Scottish league, how to differentiate between base-level bias and “team specific” bias. eg referees being naturally disposed towards any home team, and or,….referees being naturally disposed towards any perceived “stronger” team. I’m guessing it’s more difficult for any weaker team to get given a decision that goes against the perceived “natural order” of things.
As many others, and trying to be analytical I’m pretty sure our team has not been on a level playing field, but would love to get some actual metrics on it. You never know, it may turn out that our perceptions are entirely wrong!
Keep up the great work. Lovin your stuff.
Thank you. Open to constructive feedback to make it more robust. it isn’t perfect as i’d like to adjust sending off impact to account for time of game and game state (ie score at the time). If anything this framework will underplay the positive impact on decisions that go the way of Celtic and TRFC as dominant sides versus weaker sides but that is probably better than over stating on balance. But again this cannot be perfect without significantly more data and because no model is.
Agreed, no model is, (which I guess is one of the reasons it’s all so fascinating) but well done on farming it out to others prior to getting any stats in. I can’t think of a better way of doing it and am smacking my lips in anticipation at the results!
I look forward to your articles. I can understand John McGhee`s anger but ,IMO, FACTS are a far effective.
James A Barr
far MORE effective !!!
What about a referee’s stats table , basic data such as sending offs for / against – penalty’s awarded for / against etc , be interested to see a certain teams stats.
You will be doing Scottish football a service no matter the findings.
This will be interesting and not in the way you expect, I would imagine. A ‘separate entity’ referee is a great idea though.
But I have a few questions. Why wasn’t this needed over the past ten years? What happened to the boasts that ‘Sellick pyoor run Scoatish fitba’?
The quadruple treble – did the referees stop ripping the arse out of decisions just for those years?
When a decision goes celtc’s way, like Julien’s header in the league cup final, where does this stand in the grand conspiracy?
How badly has the failure to achieve 10 affected the celtc support, that the tired old conspiracies resurface?
If you were to review last season, how badly does Shane Duffy doing Shane Duffy things skew the data?
Thanks Pat for taking the time to reply. I have been wrestling how to do this for a while and only recently came across the article on impact of sendings off. That got me thinking about expected points which is a concept i already use. Then it was a case of putting a few pieces together. So this framework can be used from here on.
But your slightly defensive post says a whole lot more about your attitudes than mine. My hypothesis is that the impact of poor game changing decisions across a season will be neutral because that is what the experts in the media and the ex refs tell us.
Conveniently ignoring the points about the quadruple treble and the previous 10 years ?
I’m not sure I can help you. You’ve utterly misunderstood the point of the post.
Haha ok. I haven’t at all. I read it and find sections of it interesting enough. I merely pointing out something which you engaged in…up to a point then ignored. Where was your motivation for this over the previous 10 years!?
Alan
I only recently came upon the article on the impact of sendings off (ie about 2 months ago). The fact it used an Expected Points model allowed me to connect some dots. Then i had to find a willing referee to assist. So that’s the timeline. Funny you are trying to make a conspiracy up about it!
By using the Null Hypothesis, perhaps the chi-squared statistical test could be utilised to determine the outcome by accepting or rejecting the null hypothesis.
I will hold up my hands and say i am not a trained statistician. More a trained analyst. But i agree that at some point we need to assess whether the xPts differences are outside expected range.
Happy for you to DM me to discuss the best method to do this.
Hi Alan
Great idea! And let’s hope that team bias can be truly set aside by this method. Rather than lengthen this article I hope you continue to educate us, with selected and repeated examples of the concepts you are advancing. Being trained in behavioural Science, I have a good grasp of these issues but many of your readers may struggle to fully grasp the idea behind such concepts as Expected Points or Expected Goals. Failing that a hyperlinked glossary of the main analytical tools could be included on every blog.
Only one thing worries me in your attempt to squeeze out bias from the process and that is, your English ref will need to have full games for analysis or else he will be relying on the selected highlights of the BBC, Sky, Premier Sports or BT coverage. These selected highlights have built-in bias as the BBC showed in their Rangers v Hibs coverage by including and dissecting the Porteous RC issue, but failed to include the footage of the early Lundstram foul on Doig in their highlights.
How do you plan to combat that?
Hi and thanks. No easy answer as i cannot myself watch all the matches. Indeed i do not even watch Match of the Day or Sportscene myself due to time. I rely on events popping up on my time line i’m afraid.
I am aware also of being unintentionally disrespectful to Hibs and Hearts as they seem decent challengers but again i rely on people to bring any big calls to my attention.
This would be a terrific resource, if it were to be truly useful.
The part I’d be concerned about is where the focus is on ‘game changing’ decisions rather than the overall.
Taking Hibs/Rangers – the Porteous decision was the main focus, but there were other tackles, challenges, yellow cards that influenced the game too, so how do those get measured?
Example : If Lundstram was sent off for his challenge, Porteous was only booked, but Roofe was given the penalty – how do you balance all of that out?
I look forward to seeing this
This will be a worthwhile exercise I feel Alan and one I shall be looking forward too.
The main contentious decision at the weekend was of course the sending of of Porteous at Ibrox.
I actually think it was a dangerous tackle as his foot was over the ball and he went in recklessly endangering the opponent.
Whether he connected with the player or not is irrelevant he Could have caused serious injury had he done so.
But here’s the re was another tackle in that game by Lundstram on Doig that was just as bad and he wasn’t even booked.
So they are either both reds in the same game or they are bookings .
It see,s to me both these incidents were judged in a different manner by the same ref which in itself brings his assessment into question as a ref.
Are you going to be analysing all Scottish games, or just the ones that confirm your bias?
In my experience if either Celtic or TRFC fans feel a decision has gone against them then social media is awash with opinion, stills and video. Therefore i will probably pick it up. However, i don’t watch all the games only the Celtic ones, and i don’t watch Sportscene. I am happy for eg Hibs and Hearts fans to highlight contentious decisions also or indeed any SPFL club – ultimately it should have a bearing on the title though.
I’ve always been suspicious of myself and other Celtic supporters screaming bias. Please, please can we all try to highlight as many contentious decisions as we can from all games. The less targeted the study the better the result.
Agreed. Confirmation bias is difficult to recognise never mind set aside.
Agree – all help in good faith appreciated