Chat about Total Quarterback Rating
Welcome to SportsNation! On Friday, we continue the Year of the Quarterback when we have ESPN's Production Analytics team stopping by to explain the new Total Quarterback Rating developed by the group. The new metric will be officially unveiled Friday at 8 p.m. ET on ESPN.
As the 2011 NFL season is set to start, the Total Quarterback Rating (or Total QBR) attempts to become a more encompassing way of evaluating a quarterback's play and replace the old passer rating which was implemented in 1973. Not surprisingly, Tom Brady, Peyton Manning, Drew Brees and Aaron Rodgers all rank in the elite group of QBs in the new system.
Send your questions now and join the chat Friday at noon ET!
Dean Oliver (12:02 PM)
We're here. Good to see people already here.
TJ (Norwalk, OH)
Does the QBR system take into account the rankings/quality of the defensive team the QB is playing, or is every 3rd-and-1 on the 10 yard line the same for each QB?
Dean Oliver (12:03 PM)
We do not have the defensive adjustment in the base QBR. A defensive adjustment is like salt - easier to add than to take out.
Do you expect this to completely replace the old passer rating? What if the stats geeks don't embrace it?
Dean Oliver (12:05 PM)
Our idea was to incorporate more information and do it in the right way. Will it "completely" replace the old rating? Doubtful. OPS won't replace Batting Average.
"Clutchness" is garbage. All stats matter equally no matter when they occur. This is too subjective and unneccesarily skews the stats.
Dean Oliver (12:07 PM)
"Clutchness" isn't garbage. It is something that people perceive. In my experience, you also have anti-clutchness, where players play better because they are so far behind and the opponents aren't trying as hard because they cannot lose. Our advisors said that it was important as well.
Steven Mason (Dallas)
have you guys talked to any quarterbacks about how they feel about QB rating systems? Do they like them? Find them useful? Or do they just ignore them?
Dean Oliver (12:08 PM)
On tonight's show, you will see how some QBs feel about the current system. We have tried to address various concerns that existed about the NFL Passer Rating.
Does the new system take dropped passes into account, or the skill of the QB's receivers and blockers?
Dean Oliver (12:10 PM)
First, I will refer people to the Guide that came out yesterday. That does talk about drops. But the broader question of accounting for skills of the teammates -- yes, we looked at that a lot. Ben Alamar has done this sort of thing before and he helped a lot in setting up systems of dividing credit among the different teammates.
How do you keep track of all the stats necessary to compute the rating? How many people are involved?
Dean Oliver (12:12 PM)
ESPN has a video tracking team of about 15-20 people. They watch games either live or a little delayed and, after getting training from people like Trent Dilfer, were able to identify a wide range of additional pieces of data. This is pretty substantial work by them. You'll see that come out in various ways this year.
Sam (De Pere, WI)
I know QBR is supposed to prevent quarterbacks from benefiting from inflated yardage by way of throwing lots of short passes for large YAC numbers, but does the stat in any way factor in how a quarterback's throw can set up the receiver to have lots of open field for a big run after the catch? If so, how does it accomplish that? If not, why not? Thanks.
Dean Oliver (12:14 PM)
The analysis we did to break down credit for QBs vs receivers was meant to do this well over a lot of plays. Some passes are "more on target than others," as you refer to. Our work involved trying to identify on average how much QBs were doing to help their receivers get that yardage.
So is Matt Ryan considered an elite QB?GO FALCONS
Dean Oliver (12:16 PM)
Ryan did well with our work. He has been successful in running the offense. This past year definitely had him as Pro Bowl level. "Elite" is a word I personally reserve for truly special players. Ryan wouldn't meet that... yet.
Sam (De Pere, WI)
Does QBR draw a distinction between a pass that gets intercepted because the quarterback missed the receiver and one where he hits the receiver on the numbers but the receiver bobbles it to a defender? It seems pretty apparent that the QB has a lot more responsibility for the first scenario than the second.
Dean Oliver (12:17 PM)
Yes. We not only thought of the obvious cases but did analysis to consider the more grey cases.
Andy (B'ham WA)
How do the new QBs in the NFC West, Kolb and Jackson, compare to each other and other NFC Qbs using the QBR? Jackson has better stats, except a -2% in comp%.
Dean Oliver (12:19 PM)
Neither looks particularly good at this point, ranking below average. Kolb has had some issues throwing into traffic and having a lot of defended passes, as we've charted them.
Why isn't Philip Rivers in the elite tier? Care to give an explanation for that one?
Dean Oliver (12:21 PM)
Rivers is indeed near the top over the last three years. Last year was down a bit, but his performance over the whole length of time is reflected in his rep.
Josh (Washington, DC)
Does the QBR have a higher or lower adjusted R-squared value than the old passer rating when regressed against wins?
Dean Oliver (12:23 PM)
The answer to this one will be on tonight's show. We didn't build it to be the same as win percentage, but there should obviously be a relationship with QBs being "coaches on the field" in some or many cases.
Dean Oliver (12:25 PM)
Just a reminder that the guide to Total QBR is online athttp://espn.go.com/nfl/story/_/id/6833215/explaining-statistics-total-quarterback-ratingThis will cover some questions.
how consistent is the video tracking team? it seems that different people (even with guidance) could enter the data slightly differently, greatly skewing the stats over time. Over time, each person entering data may also perceive different things, etc, and that could cause data-consistency problems.
Dean Oliver (12:26 PM)
There are mechanisms to avoid inconsistency issues. Multiple sets of eyes are used to look at the games. They reconcile the more difficult calls.
T.J. (Boston, MA)
Are there any arbitrary minimums and maximums set on individual statistics in QBR as there are in passer rating?
Dean Oliver (12:28 PM)
No. If you're talking about the subtraction of 30% from completion percentage and max of 77% (or so), we do not have that. The only maximums are set by the game of football - you can't score 14 points on one play, for example.
Greg (North Carolina)
Out of curiosity, did you apply the new metric to the college performances of incoming QBs? It would be interesting to see how Newton, Gabbert, Mallett, etc performed.
Dean Oliver (12:29 PM)
Down the road, this is something we are interested in. Not for now.
Michael (San Diego, CA)
Is this approach to rating a QB at all inspired by Bill James' sabermetric movement?
Dean Oliver (12:31 PM)
I think we can say that Bill James and Pete Palmer's work in baseball inspired a lot of people. Knowing both of them and their work for 30 years, I continue to go back and read things they said.
Alok Pattani (12:32 PM)
Hey guys, I'm going to take over for Dean now. Keep the good questions coming!
Michael (San Diego, CA)
Are you planning to apply these concepts to other positions? Is a Total Free Safety Rating on the way?
Alok Pattani (12:34 PM)
We have designed a framework that we think accounts for the various situations in the game of football well. This system was used as the foundation for the Total QBR, but our plan is to use it for other positions down the line as well.
Won't a QB who is in games that are close be rated higher than a QB who's team dominates?
Alok Pattani (12:39 PM)
This was definitely a concern when we were designing the system. The way we've handled the clutch weight does not give excessive weight to late/close situations. The most important plays in the game - whether it was close early or close late - are given more weight than those when the score was out of hand. Late and close is weighted more than early and close, but the system is fair to QBs who have been in both situations.
Mark (St. Louis)
This is my comment. The system should be designed so the best QBs are around a 90 rating in a 100 scale. If a lot of the all-pros are in thre 60s, that's just counter-intuitiuve.
Alok Pattani (12:43 PM)
We've created the 0-100 rating so that it corresponds roughly to a percentile ranking of game-level performances from worst (near 0) to best (near 100). So you can get a score of 90-100 in a game, but having that over the course of the season is pretty tough. Just like hitting .400 in baseball is reasonable over the course of a month, but very difficult for a whole season.
Michael (San Diego, CA)
How can I get a job working for ESPN's Analytics team? It must be a blast!
Alok Pattani (12:45 PM)
If you go to espncareers.com, you'll see all openings in our Stats & Information department. You can also apply to the group that does video analysis of NFL games to provide the data off of which our rating is based.
How is Jay Cutler a below average QB. This system says that he is
Alok Pattani (12:47 PM)
Cutler is an interesting story. Our system did have him as below average last year in the regular season (he took a lot of sacks!), but he was about average the year before and in the top tier in 2008. It'll be interesting to see how he does in the coming season.
In regards to interceptions, dropped interceptions, etc., how much does down-and-distance, time, and game situation factor into the effect these have on the rating?
Alok Pattani (12:52 PM)
Down, distance, field position, time, and other aspects of the game situation are factored into our evaluation on every play. This is part of our "expected points" model that is the foundation for saying how good or bad a play was for the offense.
Is it possible to extrapolate QBR over players from a past generation (ie Bradshaw, Montana, Marino) or is it too engraved in today's pass-heavy game and situations? Either way, how would it rate a player like Bradshaw whose team won a lot yet he did not have phenomenal pure statistical numbers?
Alok Pattani (12:55 PM)
As of now, the system is set to work based on play-by-play data that our video analysis team has gathered since 2008, so we can't do years before that as easily. But we are definitely interested in historical QB ratings, and it's something we hope to look at in the future.
I just hope this doesn't just add fuel to the never ending Manning vs Brady debate.
Can we see a top ten list of Qb ratings from last year, in the new TQR?
Alok Pattani (12:58 PM)
Both of these - and a lot more - will be covered on tonight's SportsCenter Special where we reveal the Total QBR and the results. Tune into ESPN at 8pm ET to check it out!
Alok Pattani (1:00 PM)
Thanks a lot for all the great questions, everyone! We hope you enjoy the Total QBR and its use on various ESPN platforms this coming season...