Upon further review, replay officials lack consistency

Every so often an NFL coach channels postgame frustrations into a priceless tirade against the guys in stripes.

Baltimore's Brian Billick once implored the league to "dump" instant replay, this after saying he could no longer "do the company line" after years of trying.

Bill Cowher, incensed by a running-into-the-kicker penalty that doomed Pittsburgh against Tennessee, called the ruling's impact "ludicrous" before daring the league to "fine me if you want."

Seattle's Mike Holmgren, famously critical of officiating in Super Bowl XL, once accused a replay official of being "out getting a hot dog" instead of challenging a bogus Dallas touchdown.

While postgame outbursts typically carry more entertainment value than intellectual fiber, an unofficial review of instant-replay data suggests Holmgren's hot-dog theory tastes better than the typical ballpark frank.

Some replay officials are far more likely to challenge calls in the final two minutes of halves, raising questions about how the NFL applies instant replay during the game's most critical moments. In the most extreme example, replay official Dick Creed has challenged calls about eight times as frequently as Bob Boylston over the last three seasons.

"No one I know of has ever made an issue of that or even brought it up," a high-ranking AFC executive said. "That is interesting. Off the top of my head, I would hypothesize that some replay officials are better than others."

The executive identified Mark Burns as a respected replay official. Burns has issued 13 challenges in 44 games over the last three seasons, working mostly under referee Jeff Triplette.

"Those really aren't statistics that I keep," NFL officiating director Mike Pereira said. "Quite frankly, I would not attach any value to them. When a challenge comes or a review comes, we ask if it's a reviewable play and judge whether the right decision is made."

Some games produce more close calls than others, yes, but it's also fair to ask about potentially inconsistent standards for review. Might some replay officials be less inclined, even subconsciously, to challenge referees they respect or those with strong personalities? Might some replay officials issue additional challenges as insurance against missing an important call?

The league is not asking those questions.

"The one nice part about the replay system is the ref gets to make the final decision," Pereira said. "The replay guy is truly what he is, his assistant. I've never run into a situation where a guy is afraid to (challenge a referee)."

Replay officials raised 28 percent of all challenges since 2004, according to NFL totals.

The unofficial figures were much lower for referee Gerry Austin (13 percent, or 6 of 46 challenges coming from the booth) and much higher for Walt Anderson (35 percent, or 18 of 52 challenges). Replay official Hendi Ancich accounted for about one-third of referee Peter Morelli's league-high 65 challenges over the last three seasons.

Referee Bill Carollo faced 42 challenges from head coaches over the last three seasons, but only three from Boylston. The ratio was 23-3 over the last two seasons for referee Tony Corrente and replay official Bob Mantooth.

Ancich, and fellow replay officials Jim Blackwood and Ken Baker, have challenged 55 calls in about 130 games since 2004. Seven other replay officials -- Boylston, Dale Hamer, Larry Hill, Al Hynes, Bob Mantooth, Bobby Skelton and James Wilson -- have challenged fewer calls in more than twice as many games.

The numbers are unofficial, having been culled from 768 regular-season NFL gamebooks since 2004. ESPN.com counted 192 challenges by replay officials during that span, four short of the official NFL total. The league does not release statistics relating to individual crews.

Most referees have worked almost exclusively with the same replay officials over the last two seasons, part of a plan to foster continuity.

For Pereira, the bottom line is that referees erred only six or seven times in 311 total replay challenges last season.

"To me, replay is a second chance for officials to take a look at it," Pereira said. "If in their mind it's not enough to overturn, I don't consider that a mistake. A real mistake is when we overturn something that shouldn't be overturned. That is the big mistake."

The frequency of booth-initiated challenges in the final two minutes has not produced a dramatic increase in reversals. Unofficially, Morelli reversed only 2 of the 22 challenges raised from the booth since 2004. Referee Mike Carey reversed 5 of 14. Bill Vinovich reversed 2 of 12. Larry Nemmers reversed 6 of 18. Ed Hochuli reversed 3 of 10. Corrente did not reverse any of six challenges.

Holmgren hurled his hot-dog harangue at Skelton, whose non-challenge allowed Keyshawn Johnson to score a touchdown without landing inbounds. The play helped Dallas force overtime, and the Cowboys won the game. And if a more aggressive replay official had been working the game?

Pereira is right to focus primarily on the bottom line. But there is no advantage to ignoring data that could demonstrate inconsistencies.

The precedent is there. Some crews have called illegal contact and defensive holding far more frequently than others. The subject gained traction on the competition committee, leading to a "point of emphasis" on that aspect of the rulebook for the 2007 season.

The replay numbers are far from conclusive. That shouldn't disqualify them from review.

"If nothing else, it makes people ask questions," the AFC executive said. "Are they anomalies? Are we seeing trends?"

Mike Sando covers the NFL for ESPN.com.