Illusion of perfection

Automated detection technology has been invading the sports world. From the "out" call in tennis to the "offside" call in football (soccer), technology is gaining traction and replacing human judges who have traditionally made these calls.

Here is an example of an "out" call in tennis. The technology collates video footage to reconstruct the landing location of the ball. The simulation is animated and shown to spectators as evidence of an "out" call.

 

In football, a scoring player is judged "offside" if s/he is positioned behind the last defender (excluding the other team's goalie) when s/he receives the ball from a teammate. A goal is annulled if it is scored from an "offside" position. The spirit of the offside rule is to disencourage teams from parking an attacker in front of goal at all times. The "offside" call has an outsized impact on match results since football is a low-scoring game. It's a hard call, requiring accounting for the positions of three players - the attacker who scored the goal, the player from whom s/he received the ball, and the last defender. We freeze frame the moment the pass to the attacker is made, then a "calibrated line" is drawn to show the edge position of the last defender, finally, the attacker is called offside if any part of his/her body is over the line.

Defenders often move up and down as a unit, and so the positions of multiple defenders may need to be examined to find the last defender. Unlike the tennis line calls, in which the ball is found next to the line, there may be considerable distance separating the scorer from the last defender.

***

The advantages of using technology are clear. Technology applies a consistent, repeatable, testable process to come to decisions; and thus can be considered fairer. Vendors suggest that technology is also more "accurate". For example, the creator of Hawk Eye technology claims that a tennis ball's location can be determined to within 4 millimetres (the ball is about 67 millimetres in diameter). [link]

Nonetheless, some players have doubts about its accuracy. A few days ago, the German tennis star, Alexander Zverev, questioned a ball judged to be "in" by the automated detection technology. He pulled out his phone, mid-game, to take a picture of the mark on the ground, and later posted it to social media.

Because the tournament is played on clay, each ball leaves a mark on the ground. So in theory, every call made by the technology can be checked against ground truth. Zverev offered up an example of an error as the mark showed clearly that ball landed out.

This type of reality check is not possible on surfaces other than clay.

***

How does the margin of error factor into these calls?

The short answer appears to be it doesn't. The margin of error is treated like it is in a poll, and not like in air travel.

The animation shows a single ball location, without indicating the margin of error. Think of this circle as the technology's guessestimate for the true landing spot of the ball (such as that given by Zverev's snapshot). The "out" decision is based on the estimate while the margin of error adds information without threatening the decision. That's also how poll results are reported in the news media. Trump's approval rating would be reported as under 50%, even if it were at 49% with a margin of error of 3%.

In other scenarios, decision-makers let the information about the margin of error change their decision. Think about how airlines show the flight time to fly from one city to another. We all know it's heavily "padded". An hour-long flight may be shown as requiring two hours. One can use the margin of error around the average flight time to decide how much extra time to add. One would add some mutliple of that margin, calibrated to achieve a certain probability of coverage.

When applied to the tennis calls, one can add some multiple of the margin of error as another circle that envelopes the inferred ball location. If any part of the augmented ball touches the line, the ball is considered "in".

***

How to accommodate measurement uncertainty also troubles offside calls in football. Watching a bit of football lately, I am amazed at how many goals are overturned because the VAR says a finger or a toe was in an "offside" position.

Here is an example:

A goal was scored, but the striker was judged offside because according to the VAR, part of his ankle was beyond the last defender.

It's hard for me to believe that the VAR technology is capable of making such fine distinctions. Remember we are looking at a reconstruction of the scene, a composite image formed by fusing many images from different cameras. There are no sensors pasted to the fingers or toes of these players.

For me, these toetip decisions don't align with the spirit of the offside rule. Some type of margin of error should be incorporated into the decision rule. How much depends on whether they want more scoring or less.

Comments

[Antonio]

Another of your wonderful posts.
Besides what you said, I note that no focus is posed to the launch timing. Even if I can argue that the uncertainty is minor because the ball
is at the feet of the player launching the ball, I suspect that one can move one or two frames forward or backward to determine an offside position or an in-side position.
It's the same situation as in linear regression, where xs are supposed fixed and only ys are supposed affected by errors.