How QBR has measured Rodgers, Brady, and Newton

Back in 2011, we developed Total QBR to provide fans a more comprehensive tool to measure the efficiency of NFL quarterbacks. We improved on previous quarterback metrics by:

  • Accounting for the context of each play (down, distance, field position)

  • Including more than just pass attempts (rushes, sacks, penalties)

  • Using video tracked data to divide credit between quarterbacks and their teammates (air yards, drops)

  • Putting it all on an easy-to-understand 0-to-100 scale (50 is average)

What's not to like?!?! (That was a rhetorical question.)

We see the questions and critiques, but consider a recent example for why we wanted to create QBR: Marcus Mariota ran for an 87-yard touchdown on Sunday. Passer rating doesn't even acknowledge the play happened. QBR does. Like it or hate it, at least it accounts for all that a QB does. Now in our fifth season of telling stories using QBR, we've learned a lot about what fans like and don't like about the metric, and more importantly, what needs further explanation. It is important to note that the single number is including every play of every game, evaluating it in an objective, consistent way and taking into account all that a quarterback does to help his team.

What we can do to help explain QBR is find stories in which its various components are uniquely relevant. Below, you'll see that we've broken out three star-quarterback case studies from the 2015 season, all focusing on a different aspect of QBR. This should help you see the level of detail in QBR and why it is a valuable tool in measuring quarterback performance.

Aaron Rodgers: More than just a passer

About a week into training camp, Green Bay Packers quarterback Aaron Rodgers tweeted, "Can't wait to see what my Espn QBR was from practice today. Glad they factor in clean handoffs and walk thru td passes b/c I was on point."

Although that tweet was likely sent with more than a hint of sarcasm, the reigning MVP should appreciate QBR because of the way that it has captured his impact this season, a down year for him. Rodgers is one of the most accurate quarterbacks in NFL history, but this year he is on pace to post the lowest completion percentage (61 percent) and second-lowest NFL passer rating (97.3) of his career. He currently ranks 24th among qualified QBs in completion percentage and eighth in passer rating.

Like many analysts, QBR also sees this as a down year for Rodgers -- he has a 69.4 QBR this year compared to a 77.1 rating over the previous four seasons (trailing only Peyton Manning in that span). But Rodgers is still rated as the fifth-most efficient quarterback in the league because of the way he influences the game on plays other than passes.

He has contributed more expected points to his team's scoring margin on rushes and penalties than any other quarterback in the NFL. After including the negatives from sacks, Rodgers has the third-best QBR in the NFL on plays that are not directly accounted for in traditional passing measures (sacks, rushes and penalties).

To further understand Rodgers' entire impact, let's examine his game on Dec. 3 at Detroit. By traditional stats he had an "OK" performance: 24-of-36 pass, 273 yards, two touchdowns, one interception and a slightly above-average 96.2 passer rating (14th-highest in the NFL that week). Rodgers' Total QBR was 93.1, fourth-highest of the week, because of the additional contributions he made to help lead the Packers to victory.

Before the amazing 61-yard Hail Mary to end the game, Rodgers scrambled for 28 yards and drew a number of impactful penalties. With 3 minutes, 12 seconds remaining, he had a 17-yard scramble touchdown on third-and-11 to pull the Packers within two points. He also drew two defensive pass interference calls, including a 40-yarder on second-and-21 late in the first quarter.

Rodgers has produced big non-passing plays like this all season. He currently ranks second in the NFL in scramble yards (293), and the Packers rank second in yards gained on defensive pass interference calls (250).

Many of those yards were gained in key situations, including 12 third-down conversions with an average distance of 8.5 yards to go. Rodgers' contributions total more than 14 points (in terms of expected points added) on these third downs in which he didn't have a pass attempt, helping the Packers keep drives alive and move down the field in other ways.

Next time you watch Rodgers use his legs to pick up a first down or help draw a significant penalty, keep in mind that traditional passing measures ignore those plays, while QBR takes into account how much they help the team.

Tom Brady: How much credit does he deserve?

In Week 5 against the Cowboys, New England Patriots quarterback Tom Brady connected with wide receiver Julian Edelman for a 59-yard touchdown. The ball traveled 18 yards past the line of scrimmage and hit Edelman down the left sideline. Edelman ran 10 yards untouched, juked a defender and broke two tackles before gaining a total of 41 yards after the catch for the touchdown. Based purely on expected points added, that was the biggest play of the game (plus-5.6 EPA) in a Patriots blowout win.

Brady certainly deserves a good portion of the credit for the 18 yards the ball traveled in the air, but how much does he deserve for Edelman's performance after the catch?

On pass completions, the QBR algorithm assigns credit to the quarterback for both the length of the completion and the yards after catch (YAC). With every completion, there is an expected YAC given the length of throw, the location of the reception and other factors. Then using the actual yards after catch, we can calculate the yards after catch above or below what was expected, which we'll call YAC+ (a term coined by Football Outsiders),

The quarterback gets a larger percentage of the credit for the portion of the play through that expected YAC, and a smaller percentage for the YAC+. These percentages are based on statistical analysis of several thousands of NFL pass plays.

In the case of the Edelman touchdown, the expected YAC was three yards, meaning he gained 38 more YAC than expected. Brady gets more credit for the first 21 yards than the final 38, which results in him receiving 26 percent of the overall EPA on the play -- a solid percentage, but not as much as if it the pass itself had gone 40 or 50 yards downfield.

This play exemplifies a common theme for Brady this season. New England's receivers have accounted for 260 YAC+, the most such "help" for any quarterback in the NFL.

Looking only at touchdown passes, Brady also leads the league in YAC+. Brady's receivers have gained 302 yards after catch on his 31 passing touchdowns (9.7 YAC per reception), with 247 of that being YAC+.

Besides the touchdowns, Brady has 28 additional first downs than would be expected based on where he threw the ball (behind the sticks) and the expected YAC. After dividing credit on all pass plays, QBR sees that Brady's NFL-leading 3,912 passing yards and 31 passing touchdowns may be inflated by his receivers' ability to make plays.

Sure, some of the high YAC+ might be attributable to Brady playing his role in the Patriots' scheme, but it's worth nothing that Brady's receivers were close to neutral in each of the previous two seasons (a total of 18 YAC+ in 2013 and 2014). The case can also be made that with injuries to Edelman and tight end Rob Gronkowski, Brady is working with one of the weaker receiving units in the league. If Brady's new receivers are not meeting expectations in yards after catch (and they did not against the Eagles), he'll be receiving proportionally more credit for his completions than before in the QBR algorithm.

In general, the idea of dividing credit this way produces a more accurate measure of how the quarterback (not the entire offense) is impacting the success of each play. When accounting for this division of credit, it becomes clearer why Brady ranks lower in Total QBR than many other traditional quarterback statistics.

Cam Newton: Getting help from his defense

The Carolina Panthers are the NFL's only undefeated team at 12-0, putting quarterback Cam Newton at the top of most people's short list of MVP candidates. Given that, his No. 14 ranking in Total QBR may seem surprising.

But if you look at many other quarterback metrics, both standard and advanced, Newton ranks about the same, if not worse. The NFL's official passer rating has him ranked 15th, barely above the league average. So this isn't really a "QBR story," but one of perception versus reality. (Thankfully, QBR at least credits Newton for his running ability.)

When you think of Newton this season, you think of the highlights -- huge completions down the field, key first downs with his legs and, of course, tremendously elaborate touchdown dances.

Dancing aside, QBR does see these big plays that helped the Panthers score points and win games. Newton has 33 action plays that rank in the top 5 percent of plays this season in terms of QB EPA (sixth most in the NFL). But if you look on the other end of the spectrum, Newton has almost as many plays that fall into the bottom 5 percent -- 29 big negative plays, seventh most in the NFL.

Focusing on Newton's 12 turnovers (the absolute worst of those negative plays), and more specifically what has happened after them, shows why they may be overlooked when discussing his MVP candidacy.

Opponents would be expected to score about 25 points given the field position after those turnovers, including eight drives that started inside the Carolina 40-yardline. But Carolina's defense has stiffened on drives following Newton turnovers, allowing 13.3 yards per drive, no touchdowns and only 12 points. The Panthers have also gotten lucky with three missed field goals by their opponents on these drives.

The defense has essentially bailed out Cam after his turnovers, and these types of stops have been instrumental in many of the Panthers' close victories. For some perspective, Oakland's Derek Carr has also committed 12 turnovers this year, but the Raiders have given up 53 points after those.

As you can see, Newton's less-than-elite QBR actually points to an underrated part of this undefeated team -- the defense. With average support, Newton's QBR would typically translate to about seven or eight wins.

But the Panthers are second in the NFL in both QBR allowed and defensive efficiency, with only a couple below-average games in each category. That's the bigger reason why they are undefeated.

To be fair, Newton has played well of late (82.1 QBR since Week 9) and in clutch situations (league-high 89.2 QBR when the game is within one score in the fourth quarter or overtime). If he keeps playing at that level, he will obviously continue to rise in the QBR rankings and bolster his statistical case for MVP.

But the point remains that without an elite defense, Newton and the Panthers probably wouldn't be 12-0 this season. And if he and the Panthers weren't 12-0, he probably wouldn't be the leading NFL MVP candidate he's viewed as.