Football referees and Confirmation bias

[Galteeth]

So for those of you who like football, you have undoubtedly heard the calls from media sources that the debacle in the Packers-Seahawks game was the final straw proving that the NFL has to find a way to get the regular referees back on the field.

While I agree that monday night was a debacle, the thought occurred to me: Are the replacement refs actually significantly worse then the regular refs? After all, it seems like even before this, there was some complaint about a game changing botched call every week. Botched calls (as well as better technology) were the impetus for instituting replay. If you are expecting the replacement refs to be bad, every perceived bad call will reinforce this belief. Has anyone done a statistical comparison of the new refs versus the old? Given the limited number of penalties per game, is there even enough data yet to draw a statistically significant conclusion? My observation (the packers-seahawks game being an exception) was that the new refs were doing about as well as the old refs, with perhaps slightly less emphasis on db-receiver contact, and other "roughing" type of penalties, on par with the way the game might have been called 15 years ago.

 

[russ_watters]

While I agree that monday night was a debacle, the thought occurred to me: Are the replacement refs actually significantly worse then the regular refs?

Yes.

After all, it seems like even before this, there was some complaint about a game changing botched call every week.

That doesn't make sense. What happened in the Seahawks/Packers game was the worst call to happen this season, at least - a season that is three weeks old. Is your memory that short?

Botched calls (as well as better technology) were the impetus for instituting replay.

Yes. And? Replay significantly lessened the impact of botched calls...except when the replacement refs botched the replay analysis, of course.

ESPN had a top-ten list of botched calls on last night. Most were worse than Monday night's, but all that I can remember, save for the USA/USSR Olympic basketball debacle could have been fixed with replay/electronic coordination -- which didn't exist for them. So yes, replay has significantly reduced botched call impact. Not sure how that is relevant here, though.

If you are expecting the replacement refs to be bad, every perceived bad call will reinforce this belief.

Perhaps, but the difference here is so stark I don't think it is reasonable to cite confirmation bias.

Has anyone done a statistical comparison of the new refs versus the old?

Comparison of what? Bad calls are only part of the problem. They've also had trouble with the down-to-down running of the game, getting bullied by players/coaches and losing control of the demeanor of the players. Just spotting the ball for the next play has been problematic. Perhaps then the easiest measure might be average time of game this year vs last. These problems are more pervasive than the missed calls and at least as significant.

Given the limited number of penalties per game, is there even enough data yet to draw a statistically significant conclusion?

I don't know -- what exactly are you trying to measure in the penalties? Just which ones were bad?

My observation (the packers-seahawks game being an exception) was that the new refs were doing about as well as the old refs, with perhaps slightly less emphasis on db-receiver contact, and other "roughing" type of penalties, on par with the way the game might have been called 15 years ago.

I disagree. And you can't filter out the most relevant piece of data in the analysis or the other measures of competency if you want your analysis to be meaningful!

 

[Ryan_m_b]

Out of interest does American football use the convention of off-pitch refs who can replay and use other goal line technologies to make a decision or is it more like everywhereelse football in that the on-pitch ref's decision is final and he/she does not get to request replays et al?

 

[BobG]

Comparison of what? Bad calls are only part of the problem. They've also had trouble with the down-to-down running of the game, getting bullied by players/coaches and losing control of the demeanor of the players. Just spotting the ball for the next play has been problematic. Perhaps then the easiest measure might be average time of game this year vs last.

This is a pretty good measure of how good they are compared to the NFL referees. The game changing, "bad" calls don't happen frequently enough to compare.

Plus, the confusion over rules. Didn't charge a timeout to a team in the last 2 minutes for an injury timeout, effectively giving them 4 timeouts (not a great rule, in spirit, but it does keep players from faking injuries just to stop the clock). Gave the 49ers two challenges when they had no timeouts (the penalty for losing a challenge is to lose a timeout, so need a timeout to lose).

The latter illustrates an important point. It's not disasterous to the game (big picture) to have a bad call stand, even if it does change the game result, hence there are times when the coach can't even challenge a bad call.

How do the replacement referees actually affect the game? They increase discussion about the NFL. The excitement doesn't end on Monday. People talk about last weekend's games all the way through the week to Thursday - which is usually the next NFL game.

Replacement referees don't hurt the NFL any more than having fans argue about who is really the #1 college team in America is. Simple, fair games don't increase attendance (which is usually sold out already) and bad game changing calls don't decrease attendance (which is still usually sold out).

Incidentally, part of the problem has to do with NFL referees association with college football. Most of the big conferences have NFL referees as referee supervisors, consultants, etc. As a result, none of the major conferences have allowed their referees to referee NFL games. So, the NFL isn't just stepping down one level when it comes to refereeing. They're having to step down quite a ways in ability to get replacement referees.

This is good for NFL referees (perception wise, at least). It makes them look like there's a huge difference between them and any other referees, which isn't quite true. But there is a huge difference between NFL referees and small college referees/high school referees.

 

[russ_watters]

Out of interest does American football use the convention of off-pitch refs who can replay and use other goal line technologies to make a decision or is it more like everywhereelse football in that the on-pitch ref's decision is final and he/she does not get to request replays et al?

We use off-field refs and instant replay for some calls. Coaches are allowed 2 challenges a half, i think. And scoring plays are automatically reviewed.

 

[Gale]

Comparison of what? Bad calls are only part of the problem. They've also had trouble with the down-to-down running of the game, getting bullied by players/coaches and losing control of the demeanor of the players.

I pretty much agree with everything you said, and you and BobG are right to note the length of the games as an indicator of the inexperienced refereeing.

And as you noted, one of the biggest problems with bad refs is the way the teams and coaches react to them. It ruins the whole game when you're playing the refs instead of playing the other team. Bad calls tend to get evened out throughout a game (though obviously some bad calls hurt more than others...) but inconsistency in the calls makes it hard to play good football.

Regarding the Packers/Seahawks game:
That was just one example of a really terrifically bad call that happened to get a lot of attention because the game was on Monday Night Football (most NFL games are on Sunday, and most people can only watch one game at a time... but there are a couple prime time games a week that get undivided attention from all NFL fans.) But as its been pointed out, that hasn't been the only big mistake of the last few weeks. And anyway, its the overall quality of the game that has been degraded more so than just a screwed up call here and there.

 

[Gale]

We use off-field refs and instant replay for some calls. Coaches are allowed 2 challenges a half, i think. And scoring plays are automatically reviewed.

Worth noting is that most plays in football are "replayed" for the audience. The coaches can often see the instant replay of many plays inside the stadium before they decide whether they'd like the official's call reviewed, so long as they challenge before the next play starts.

Also, the coaches can't challenge during the last 2 minutes of each half. During those minutes, all calls are reviewed by the team of refs "upstairs".