Dirty Laundry: Spot challenges

We've covered this week's spate of controversial officiating calls, and more is on the way. So this week, I thought we would use Dirty Laundry to follow up on Jim's insightful request:

In light of Lovie Smith's now-famous failure to throw the challenge flag last weekend and his previous failed challenge, how about giving your stats guys an opportunity for extra credit? I'd be curious to see a recent history of NFL challenges broken down by category (what was challenged) and the success rate.

The reason I ask is because of Lovie's failed ball placement challenge. I've been watching NFL games since the instant replay system was instituted (and long before), and I don't think I've ever seen a coach successfully challenge the spot of the ball. I imagine it has to do with the absence of landmarks on the field to use for frame of reference. I'd be shocked if that yellow first down line is on their feed under the hood.

It would also be interesting to hear from an official on the topic. But my guess is that ball position challenges are only successful a tiny percentage of the time and NFL coaches would be best advised to hold the flag in those circumstances. Even when it seems obvious that the ball should be moved.

First, Jim, NFL officials are not available for interviews. Second, I actually use statistics the NFL provides to the media for our weekly Challenge Tracker (which, never fear, will return next week). The NFL also breaks down challenges into several categories. None of them fully reflect spot challenges, but I think we can still get a pretty good idea from the numbers we do have.

The chart below shows the type and result of every 2010 challenge based on how the NFL categorizes them. I included the preseason to give us a bigger sample size, figuring that challenges are among the few legitimate crossovers from the preseason to regular season.

"Spot" calls can fall into any number of categories based on the situation. It could be to determine whether a first down was achieved or whether a runner was down by contact. First downs are probably the closest thing to what you're asking about, and indeed, there has only been one overturned challenge in 13 attempts this season.

In 2009, nine of 33 first-down challenges were overturned.

In 2008, 17 of 42 first-down challenges were overturned.

That makes 27 of 88 over the past 2 1/2 seasons, or 30.6 percent. So the answer to your question, Jim, is that spot calls are in fact overturned but recently they have been at a more limited rate than other types of challenges.

Specific to Sunday's Chicago Bears game at Soldier Field, however, Smith's odds were higher. The chart shows there have been 19 challenges to whether the ball broke the plane of the end zone. Eight have been overturned. That's what Smith was asking officials to consider when he challenged the spot of Earl Bennett's 48-yard reception: Whether the ball crossed the plane before he was tackled.

Replays showed it was a poor challenge from the start, so let's remove it from our sample and note that nearly 45 percent of the other calls have been overturned. Those are the odds Smith would have been facing had he challenged the next play, a Jay Cutler fumble that came after he appeared to reach the ball over the goal line.

What to make of these numbers? Obviously, it's notable that spot challenges are sometimes overturned but generally not at the same rate as other categories. More than that, however, I think it's interesting to look at the frequency of challenges. By far, the biggest challenge is whether a pass was complete or not. Fumbles are second, followed relatively distantly by everything else.

For reasons I'm sure you'll discuss in the comments section, most coaches are saving their challenges for those types of plays.