If you're the kind of person who likes to see at least some sort of quantifiable data before you make a judgement, analyzing officials is hardly your best bet. Where officials are concerned, it's all emotion and anecdote: Referee X calls too many fouls, Referee Y defaults to the defense on charge calls,
Ted Valentine Referee Z likes to be the center of attention.
These reputations may be deserved, but they are nonetheless earned in heated one-off moments that tend to stick out in a observer's brain. They are not necessarily representative of an official's entire body of work -- all of the games, all of the possessions, all of the calls -- in the various conferences and environments under which any given dude might work.
The problem is that we don't really know. Beyond fouls, we don't have reliable statistics for officials. We can't dig into the same kind of detailed information we have for players and teams. Our impressions are based on generalities.
"If we had officiating statistics, that would be different, if only slightly. If we knew that a certain official had the highest accuracy rate in the land, we could put a missed call in context. However, that’s not the world we live in, and nobody is going to take the time to monitor whether Verne Harris likes to call the arm bar in the post, so we are left with 'officiating in [name of conference] sucks.'"
Ken Pomeroy, writing above, is nonetheless willing to try. On Tuesday, Pomeroy published the results of an exhaustive study on the foul-calling habits of 350 Division I officials. He dug back into hundreds of box scores, corrected scorekeeper misspellings, adjusted for a variety of factors (which refs worked with each other, when a game took place, final margins, overtimes, fouling habits of the two teams, and so on) and emerged with a high-level view of individual officials' foul habits. Which two officials who call the fewest fouls relative to average expectation? Roger Ayers and Ted Valentine. True story.
So that's useful: If you have Ayers and Valentine on the same game, you might rightly expect a leniently called, physical affair. And if you see Paul Janssen and Jerry Heater, two of the study's stricter whistle-blowers, you might rightly expect a free throw parade. But even if you're the kind of mentally disturbed die-hard who recognizes officials in every game you see, there aren't many overarching conclusions to be culled here. In any one game, anything can happen. And as Ken notes:
Regarding the results of this study, there really shouldn't be any value judgments placed on the numbers. Whether a guy calls more or less fouls than average is just a thing. If you know who’s on the crew for your team’s next game, let this be a handy reference as to whether the game might be called tight or not. If a game has 55 fouls, it's worth referencing whether the crew was populated with foul-happy officials before accusing them of hating the game of basketball. There are games where a lot of actual fouls happen, unfortunately.
Caveats and all, his is an interesting attempt to inject some hard information into the squishiest regions of the college hoops corpus. It's something, at least. And until we arrive at that far flung universal-SportVU future, this may well be the best anyone can do. Few things are as hard as a block-charge call. Putting that call in its proper context may long be one of them.