Upgrades sharpen ESPN's college FPI model

Matt Cashore/USA TODAY Sports

ESPN’s Football Power Index was introduced two years ago as a way to measure team strength and predict performance going forward.

As with all of the ESPN Stats & Information Group's metrics, we are looking constantly for ways to improve FPI.

In 2014, we introduced preseason FPI as a way to predict team strength entering the season. Preseason FPI not only allowed us to produce FPI ratings before a single pass was thrown, but it also improved the overall accuracy of our FPI-based projections by accounting for information not fully captured in the original version.

The result? The team that FPI favored won 76 percent of FBS-vs.-FBS games last season, which was a higher percentage than almost any other system out there -- including the Vegas closing line -- and the majority of ESPN analysts.

Entering the 2015 season, we are again looking at ways to sharpen FPI and improve the accuracy of its predictions. Below are three improvements we made to FPI and the theory behind those adjustments.

Better accounting for FCS opponents

Most college football fans would acknowledge that playing four-time reigning FCS champion North Dakota State is not nearly the same as facing a bottom-tier FCS team. The Bison have won five straight games against FBS opponents dating to 2010, including four straight against Power 5 teams. That’s as many Power 5 wins as Kansas has since 2010!

So when North Dakota State headed to Iowa State last season, it was clear to the average fan that the Cyclones were not facing a typical FCS opponent. In the previous iteration of FPI, however, all FCS teams were regarded as being of the same caliber and rated below the weakest FBS team. Based on that logic, Iowa State entered the game with a 97 percent chance to win, and when they lost 34-14, the Cyclones fell 31 spots in FPI.

In a small 12-game sample, misrepresenting the actual strength of even one opponent could have a significant impact on a team’s FPI, which has a trickle-down effect on a number of FPI’s outputs -- such as future game projections and strength of schedule.

From the moment FPI was released in 2013, we knew this was a problem that had to be solved, but because of limitations with FCS data, we rolled out FPI with the intention to resolve this issue in the future.

Today, each FCS team has an FPI rating based on the final score of games dating back four years. As with FPI for FBS teams, the rating is a measure of team strength and represents the expected scoring margin against an average FBS opponent.

For example, North Dakota State enters the season with an FPI rating of plus-0.9, meaning if the Bison were to play an average (or 64th-ranked) FBS team, they would be favored slightly on a neutral field. In the coming season, there are 32 FCS teams with a higher FPI rating than the weakest FBS team and seven FBS-vs.-FCS games in which the FCS team has at least a 25 percent chance to win.

By capturing the true strength of each FCS team, we are also improving the accuracy of FPI’s FBS ratings and game predictions. With the updated formula, the team FPI favored continued to win 75 percent of FBS-only games since 2005, but predicted scoring-margin errors decreased compared with the previous system.

After accounting for the relative strength of FCS teams, Iowa State entered its home game against North Dakota State with a 66 percent chance to win. After losing to the Bison, the Cyclones fell 15 spots in FPI, and what was once a historically crippling loss was treated no differently than a loss to about the 66th-ranked FBS team.

Incorporating transfer quarterbacks

Russell Wilson is the poster child for the graduate transfer. After graduating from NC State, Wilson transferred to Wisconsin, where he led the Badgers to a Big Ten title and Rose Bowl appearance, while completing one of the most efficient single seasons in NCAA history.

Wilson was not the first player to utilize the NCAA’s graduate transfer rule (enacted in 2006), but his success in Madison began a revolution of players extending their college careers through “college free agency.”

In 2015 alone, Florida State, Georgia, Oklahoma, Michigan and Oregon are expected to have a graduate transfer competing for the starting quarterback job.

Although the success rates of graduate transfers have been mixed, it is clear that adding an experienced signal-caller has its advantages. Everett Golson, for example, heads to Florida State with two years of starting experience and an understanding of how to compete on the biggest stage.

One input into preseason FPI is the number of returning starters on offense and defense, with quarterbacks counted separately. Based on how strong the offense was last season, returning a starting quarterback could have a significant impact on an offense (compared to not returning one).

In the previous iteration of FPI, transfers were not counted as returning starters, but based on the small sample of quarterbacks tested, the presence of an experienced transfer was shown to have an impact on a team’s future success. That player still has to make a transition to the new team, however, so the impact on FPI is not the same as it is with returning a full starter.

For this reason, we are counting any transfer quarterback with at least 240 action plays at the FBS level (or the equivalent of one year starting) as half a starter. This affects a handful of teams, but it is a major reason Florida State jumped over Clemson in the latest FPI.

Offensive continuity

Led by one of the most efficient offenses in the nation last season, TCU set single-season school records for points, yards and touchdowns. With 10 offensive starters, including Trevone Boykin, and coach Gary Patterson returning, the Horned Frogs should be even better with another year of experience.

In the previous version of the offensive model, starters and previous years’ efficiencies interacted, but steps were taken to smooth out the continuity of this interaction.

In other words, if a team is returning several offensive starters, its quarterback and its coach, chances are the offense will look a lot like its offense from the previous season. Now, in the model, the number of returning starters correlates more (for better or for worse) with last year’s offensive efficiency than it had.

This same interaction was not as strong when tested on defensive starters, but there remains a benefit to returning a large number of starters on either side of the ball.

Returning to the TCU example, the Horned Frogs return 10 starters to an already strong offense, resulting in a better projected offense than in the previous model. SMU, by comparison, also returns 10 starters, but because its offense was one of the worst in the country last season, the Mustangs don’t get the same offensive boost in their FPI rating as the Horned Frogs do.

In addition to the changes outlined above, information on returning starters has been updated since the early-July release of FPI. UCLA, Utah, Tennessee, Clemson and Michigan State are a few notable teams that lost a starter to injury, suspension or transfer, according to Phil Steele.

The overall impact of these changes on preseason FPI is minor; 23 of the teams ranked in the top 25 in FPI in July remained there, and five of those original top 25 teams moved up or down by more than two spots.

The goal of FPI is to measure team strength going forward to make the most accurate predictions. After implementing the changes outlined above and retroactively running game predictions beginning in 2005, FPI remained one of the most accurate systems (correctly predicting 75 percent of games) and should be more accurate in the future.