Big Ten officiating is in the crosshairs after the events of recent weeks.
There has been as much or more attention on the seven men in stripes and the two in the replay booth as the 22 men wearing helmets and pads. Big Ten coaches and players have openly questioned calls, and many fans, with the help of technology, have dissected key decisions and voiced their opinions as well.
Bill Carollo is well aware of the issues. The Big Ten's coordinator of football officials, in his fourth year on the job, oversees all the crews, addresses all questioned calls from coaches and provides answers, even if it's just, "We made a mistake and here's why." Carollo also manages all scheduling, training, evaluation and penalties for Big Ten officials.
League policy prohibits him from commenting on specific calls, but he spoke extensively with ESPN.com this week in an effort to shed more light on how games and officials are evaluated, instant replay issues, the consequences for mistakes and the key emphasis points for his crews.
"We were having a good year," Carollo said. "Have we stumbled the last couple weeks in some high-profile games? Absolutely. Are there a couple plays that I would like to have officiated differently? Absolutely. Is it going to be perfect? There's never been a perfect game -- played, coached or officiated. Never.
"But we certainly can be consistent. That's what our goal is."
Let's dive in ...
How are Big Ten officials evaluated?
Every play involving a Big Ten officiating crew is graded. Carollo, a former NFL referee who officiated two Super Bowls and seven conference title games, has contracted active NFL officials -- not affiliated with the Big Ten -- to serve as independent evaluators. Coaches send in their game tapes to the Big Ten after each contest, and evaluators use those camera angles (end zone and and sideline) plus the television broadcasts to do their grades.
In addition, Carollo has "coaches" for each official on the field and those in the replay booth. For example, former NFL referee Jerry Markbreit and current NFL referee John Parry work with the Big Ten's head referees. Dean Blandino, who directs the NFL's instant replay program, works directly with the Big Ten's replay officials.
Game grades must be completed by Tuesday, and officials receive their grades with comments on Wednesdays. All mistakes are noted, and officials can even earn a demerit if they make a correct call from the wrong position. One goal is not just to list the mistake, but the reason for it.
Head referees communicate with the evaluators and can question grades on certain plays.
Crews average about five mistakes per game, and the numbers haven't increased in this particular season.
"There is accountability, and that's the key thing," Carollo said. "They've never been evaluated like this before. If we don't evaluate our officials, if we can't measure them, we won't improve."
What type of input do coaches have?
Carollo receives a list of plays to review from every Big Ten coach after every game (he also hears from athletic directors). He asks the coaches to take a 12-hour "cooling-off period" after the game before submitting their plays. The submissions come in mostly Sunday and Carollo responds to each one himself, providing explanations for why a penalty was called (or why not). Carollo typically watches all games in the Big Ten's television command center and notes calls that he could be hearing about later from the coaches.
Carollo doesn't directly grade his crews -- that's what the independent evaluators are for -- but he will provide his views by Tuesday night.
"I want to know what the coaches' biggest beef is," he said. "The teams get the right answers from us."
What are the key goals of replay?
The purpose, according to Carollo, is to fix major mistakes, particularly those involving scoring or change of possession. The goal isn't to review whether it's second-and-6 or second-and-7 in the first quarter.
A decision to reverse a call on the field "has to be indisputable" for the replay officials. Carollo wants any judgment calls to be made by the on-field officials.
"If [the replay official] has to go a couple minutes, there's probably some question," Carollo said. "Either you don't have the right video to prove it, or it's so close, it's so tight. I'm not sure, but I think it's a touchdown. I think it’s a completed catch. I think he stayed in bounds. Well, in replay, you can't think. You have to know. Because the premise is the ruling on the field is correct unless you have absolute 100 percent proof. Not 98 percent.
"That doesn't fly in the replay booth."
What are the consequences for mistakes?
All Big Ten officials are on one-year contracts. Carollo tentatively schedules officiating crews 3-4 weeks in advance, but nothing is final and officials can be taken off of games for poor performance.
The decisions are made internally and not revealed publicly, but "that has happened," Carollo said.
Carollo rotates crews so they see different teams and tries to mix experienced officials with younger ones. But he notes, "I try to put my best officials that are officiating well on the biggest games."
These include the Big Ten title game and any bowl game where a Big Ten crew is used.
What happens when an official blows a critical call?
"It's red-lettered and circled," Carollo said. "A real big minus to the point where one call could cost you your job, one call could cost you a postseason assignment, one call could take away the rest of the season or be a suspension."
Carollo doesn't like suspending officials for a poor judgment call and points out that if he fired or suspended every official for a mistake, there wouldn't be enough of them to work games. Some officials also are moved to different positions if they're deemed to be out of place.
Replay officials are assigned to games independently -- in the future, they may work with the same on-field crews, but not now -- are graded on every review or potential review where a play should have been stopped but wasn't.
"If they had something that was very controversial or incorrect, that would be figured into their grade, into their assignments and for sure into their postseason and if they're going to be retained," Carollo said.
Carollo is a big believer in constant evaluation, and he knows the buck stops with him.
"They should fire me if I've got the wrong guys out there," he said. "I hired them, I'm training them and I take full responsibility for them."
Are penalties tracked against certain teams?
At the end of each season, Carollo provides each Big Ten coach with a 50-page report that details, among other things, every play, every call, every replay review, all correct calls, all incorrect calls, which plays the coaches questioned (and Carollo's responses) and which crews worked the games.
As a reminder, the crews are rotated so they get to see each team multiple times.
"I tell the coaches, 'Here’s how you did in the five most popular areas for flags. You led the league in high hits. Maybe we should talk to your team about high hits. You led the league in pass interference,'" Carollo said.
What are the key points of emphasis?
Pass interference was a focal point in Carollo's first season in 2009, along with holding. But high hits have been the chief concern for Carollo and his crews the past three seasons. Carollo acknowledges he'd rather the officials throw more flags for high hits -- even if their accuracy isn't as strong as it is for other penalties -- if it helps reduce them.
"We have probably led the charge in that area, the Big Ten, as far as high hits and reprimands on players in the last three, four years," Carollo said. "The rest of the country is catching up in this area. We've made some big improvement in our most important area."
Pass interference has been spotlighted in recent weeks as coaches from several Big Ten teams have voiced complaints.
There are six categories for defensive pass interference and three for offensive pass interference. Big Ten officials will cut defenders slack for playing the ball, while those who don't will almost always be flagged for contact.
Carollo notes that some officials are "going too quick" and throwing flags because they think a defender will interfere before he actually does.
"The first eight weeks were very good; the last two weeks, not as good," he said. "Pass interference has raised its ugly head, so we're spending a lot more time on that with our trainers, with our graders and my training tapes."
What's the objective going forward?
"It's a challenging job to get consistency, and I think we’ll always have some inconsistency," Carollo said. "The Big Ten coaches have been outstanding. They understand the guidelines, they understand it's a hard job, but they just want to make sure it's consistent and it's fair.
"We're trying to work toward that."