Stats & Info: Tell Rob Ryan basic numbers lie

After learning of his firing on Tuesday, former Dallas Cowboys defensive coordinator Rob Ryan told ESPNDallas.com's Tim MacMahon the following:

"I inherited a team that was 31st in the league in defense and made them better. I (expletive) made them a hell of a lot better. I'll be out of work for like five minutes."

Given Ryan’s extreme amount of confidence and propensity to speak his mind, this type of statement comes as no surprise. But is it actually true?

A quick look at the numbers from 2010 -- the year before Ryan was hired by the Cowboys -- shows that Ryan was referring to points allowed per game (often referred to as “scoring defense”), as the Cowboys ranked second-to-last in the league with 27.3 Opp PPG that season.

Looking at the same number for the past two seasons with Ryan calling the defense, the Cowboys ranked 16th in the league with 21.7 Opp. PPG in 2011 and 24th with 25.0 Opp. PPG in 2012. While most wouldn’t interpret those numbers as befitting of a “top” defense, they would seem to indicate some improvement from 2010 when taken at face value.

But a major issue here is with how defense is being measured. Is scoring defense really a true measure of defense and defense only? Examining more in depth, there are a few reasons why that statistic could be misleading:

  • While most points are allowed by the defense, points can be scored on return touchdowns against the offense or special teams but still count towards a defense’s “points allowed.”

  • On the flip side, defenses can score touchdowns, which improves their teams’ “scoring offense” but doesn’t directly show up in scoring defense.

  • Defenses that have to face shorter fields as a result of poor offenses or special teams are likely to give up more points. This can result in them in having a higher Opp. PPG due to poor performance by other units on their teams, not the defense itself.

  • And conversely, defenses that force turnovers or snuff out opponent drives without allowing many yards often turn the field over in better field position for their offense. This helps the offense score (and the team win) but doesn’t show up in Opp. PPG.

    In summary, scoring defense does not look at the context of each situation the defense is put in and does not account for everything a defense actually does to affect the scoreboard, making it an imperfect measure of defense in football. Comparing the Cowboys' defense from 2010 to the two more recent years under Rob Ryan in some other ways provides a good “case study” of this, as many of the aforementioned factors are at play.

    First off, the yardage numbers (often referred to as “total defense”) are extremely similar, indicating that the two periods may not be as disparate as the scoring defense numbers suggest. Yardage can be even more misleading than points allowed, but limiting opponent ball movement is obviously one very important part of defense.

    Now, consider that the average opponent drive start against the Cowboys was over three yards further back for Ryan’s defenses than in 2010. This adds up to about 35-40 yards of field position over the course of a game. Part of this was the kickoff rule change, but it also reflects the Dallas offense doing less to hurt the defense the last two years, committing fewer turnovers and going three-and-out less frequently than it did in 2010.

    If two defenses give up the same amount of yardage but the opponent starts in better field position on average against one, that defense will likely give up a few more points over the course of the year due to factors mostly outside of its control. Not only that, but the 2010 Cowboys allowed seven non-offensive touchdowns, compared to an average of five in 2011 and 2012 –--there’s almost another point per game difference in Opp. PPG that isn’t really due to defense.

    Finally, the 2010 defense forced almost two turnovers per game and converted four into defensive touchdowns. Under Ryan, the Cowboys forced just 1.3 turnovers per game (26th in the NFL) and had the same number of defensive scores over two years that they had in the single year before he arrived.

    So how do we put this all together and see where the Cowboys defense really ranked the last three years? Using an advanced metric called Expected Points Added (EPA) that looks at the situations that an offense/defense/special teams is put in and calculates how that specific unit improves the team’s potential to score and prevent the opponent from scoring.

    Defensive EPA encapsulates everything that a defense does to affect the game situation while the unit is on the field, and it is not “muddied” by other factors outside of the defense’s control like many traditional defensive stats are. It is on the scale of net points, so the average is 0, with positive numbers for units that help and negative numbers for defenses that hurt. (For more on EPA, see here.)

    Below are the Cowboys' defensive numbers and ranks in terms of EPA per game over the last three seasons.

    Yes, the Cowboys have actually declined in terms of defensive performance by this measure under Rob Ryan. Part of this is how the league has become even more offense-friendly over the last couple years, as Dallas’ ranking actually improved despite the EPA number going slightly down from 2010 to 2011. But the same can’t be said for this past year, when the Cowboys' defense cost the team over five net points per game more than an average unit and played a major role in the team being left out of the playoffs for the third straight season.

    Of course, this defensive decline can’t be entirely attributed to Ryan, especially with the number of important injuries the Cowboys dealt with on that side of the ball over the last few weeks of the season as the defense went from average to worst in the league. But by measuring defense more accurately, it’s clear that Ryan’s claim that he made the defense “a hell of a lot better” is yet another case of the false bravado with limited results that was common during his short tenure in Dallas.