Total QBR was introduced in 2011 to improve upon the NFL passer rating by accounting for more of what a quarterback does, including his rushes, scrambles, fumbles and sacks. It also accounts for when things happen -- 5 yards on third-and-4 being more meaningful than 5 yards on third-and-14, for instance.
As QBR enters its third NFL season of use, we have found a few additional things to improve it, as we did last year.
The biggest change is how clutch weighting is done. Based on our experience in the NFL and our recent work on QBR for college football, we have found that emphasizing performance in clutch situations serves relatively little benefit compared to de-emphasizing performance when a game is out of reach. Good or bad performance when a game is out of reach isn't as relevant because the game isn't played the same way as when it's close or early.
Plays are "bigger" in tight games because they do change the chance of winning a lot more, but the impact of rewarding that is to de-emphasize what a quarterback did before they got to the clutch situations, even though those situations were still competitive. As a result, clutch weighting now primarily serves to de-emphasize performance when a game is less competitive; that weight is, as it was before, related to potential changes in win probability.
There were a couple of other changes to QBR to help improve the handling of very specific (and somewhat unusual) situations that we noticed.
• In a Rams-49ers game on Dec. 2, 2012, Colin Kaepernick tried to make a pitch back to a running back. That pitch sailed over the back's head, ending in a Rams recovery and touchdown. In the play-by-play that came from the NFL, this was listed as an aborted snap. An aborted snap is not what we think of as happening here, so we now chart it as an aborted pitch. Aborted snaps are usually fallen on by the offense, whereas a bad pitch is more a 50-50 ball for the offense or defense to recover. Aborted snaps can be the fault of the center or the quarterback. An aborted pitch is more likely to come from the quarterback. Making this change hurts Kaepernick more on this play and for the game, which is appropriate for what happened. In general, we know that individual plays may have a division of credit that isn't perfect, but this is one in which a simple fix can help.
• On Sept. 30, 2012, Atlanta was facing second-and-goal from the 1-yard line against the Panthers. The Panthers were drawn offsides, giving the Falcons a penalty that put them half a yard closer. But the official situation is still second-and-goal from the 1-yard line. There may have been an actual benefit to that half a yard (or less), but our methods don't see it -- the Falcons' expected points at the start of the play remained the same. In these situations, we were giving the quarterback an action play, which then lowered his QBR because there didn't appear to be a gain for the team. In these zero-yard penalties (that don't give a new set of downs), we stopped giving a play to the quarterback.
To emphasize, none of these modifications dramatically impact rankings. The modification on clutch weight changes things most at a game level, especially in close games. But over the course of a season, we see little change in QBR impact, implying that performance in clutch situations didn't dramatically affect long-term perception. The other two modifications also make very little difference.
Further, these changes don't affect QBR's status as a key statistic. Having a higher QBR than your opponent wins the game about 85 percent of the time, much higher than turnovers, yards or passer rating.
Finally, through our relationship with Football Outsiders, we have acquired charting data for the 2006 and 2007 regular seasons. This charting data allowed us to generate QBR values for those two seasons in addition to the years 2008-2012 that we already had. Peyton Manning dominated 2006 like we haven't seen in any other year, having nearly a 20-point lead over the second-place player. And in 2007, when the Patriots acquired Randy Moss and went 16-0 in the regular season, it's not surprising that Tom Brady's brilliant 50-touchdown season ends up at the top. These are just quick examples of the many stories that can be told with QBR. A few others are shown below:
• Robert Griffin III was really good when blitzed, avoiding sacks, scrambling, and throwing well, which is why he wasn't blitzed often, as Ron Jaworski highlights.
• The Cardinals' defense was tremendous at shutting down quarterbacks last year, as told by Mike Sando.
• In his rookie season, Andrew Luck's running was as effective as RG III's, and Luck was generating more from the pass game than RG III, as described by Rick Reilly.