Special to Page 2
NOTE: For more on ZEUS and its founders, check out the Dec. 18 issue of ESPN The Magazine.
The winner of Sunday afternoon's Cowboys-Giants game in the Meadowlands would claim first place in the NFC East with only four regular-season games remaining -- a marquee matchup for any NFL fan. A contrast entering the game was equally as obvious: Dallas was riding a three-game winning streak, the Giants a three-game losing streak. The oddsmakers put Dallas as a 3½-point favorite on the road -- about a 60-40 edge to win the game outright.
ZEUS™, a creation of EndGame Technologies Inc. (www.pigskinrevolution.com), is a software product that, among other things, can evaluate the performance of an NFL team's play-calling decisions in critical situations, including fourth downs, extra points and kickoffs (kick away or kick onside). ZEUS uses distributions of individual play outcomes based upon historical NFL data and the customized characteristics of the individual teams in its repeated trials evaluations.
The table below delineates its conclusions regarding the critical play calls in Sunday's Cowboys-Giants game.
|BREAKDOWN OF KEY CALLS|
|Actual call||ZEUS call||
|NYG||0||2||1:30||opp 24||4||1||end run||interior run||-1.0||4|
|NYG||0||3||4:32||own 30||--||--||kick away||kick away||0||7|
|NYG||0||4||9:44||own 30||--||--||kick away||away/onside||0||0|
|NYG||-1||4||1:06||opp 2||--||--||1 point||2 point||-2.4||10|
|NYG||0||4||1:06||own 30||--||--||kick away||kick away||0||5|
The first column shows the team with possession. The second column is the score difference (score of team with possession minus opponent's score). The next five columns should be self-explanatory to anyone who understands football. The eighth column is the actual play call made during the game followed by ZEUS' recommendation for that identical situation.
The last two columns are the quantitative part of the analysis and require a more detailed explanation. "%GWC error" is the amount (on average) the actual play call costs a team, expressed in GWC (game winning chance) units. In typical situations, play-call errors cost at most a few percent. A large play-call error might be 5 percent, and a very serious blunder might reach 10 percent. To further illustrate the concept of a serious GWC error, imagine a team facing fourth-and-goal from the 2-yard line and trailing by 2 points on the last play of the game. Everyone knows the field goal would be the optimal choice in this situation. But suppose the coach elects to run the ball and succeeds in scoring the touchdown and winning the game. Although he still might have collected the W, ZEUS will charge him with costing his team about 50 percent GWC. This is because on average, he would have won the game approximately 98 percent of the time with a field goal attempt and only about 48 percent of the time with a run (subject to the custom features of the offense and defense). To add a bit more perspective, if a team averages 6 percent GWC errors per game for an entire season, the aggregate effect is (16 times 6 percent = ) one full game that season. That's a significant sacrifice in the NFL, where one extra win can mean the difference between making the playoffs and going home.
The "confidence" column is a measure of the strength of ZEUS' result. The range is zero (a pure coin toss between the top two plays) and 10 (no realistic extraneous factor can overturn ZEUS' decision). The method of determining the confidence proceeds roughly as follows. After the correct play call for the teams is determined, the performances of the two teams are adjusted away from the norm in the direction that would be most likely to overturn the correct decision. Then the situation is reassessed by ZEUS. The "confidence" is a measure of how difficult it is to overturn the call. For example, suppose ZEUS initially says going for the first down is preferable to a punt. Four factors are adjusted: the punting team is made stronger, the punt return team is made weaker, the offense (trying to get the first down) is weakened and the defense (trying to stop the first down) is strengthened. All of these adjustments are in the direction to reverse the decision that going for the first down is correct. The resulting decision is then compared with the original decision, and a confidence value is assigned. A small number (0-3) says the original play call is close and might be overturned if the customized conditions (based upon teams' past performance) are inaccurate. Confidences of 4-6 are moderate and probably correct but still in doubt. Higher numbers mean it is very unlikely any misevaluation of the individual characteristics of the teams could lead to a reversal in ZEUS' decision. Think of a confidence of "10" this way: Even if one team is playing at the level of a mediocre collegiate squad and the opponent is the Super Bowl champion, the original ZEUS recommendation still holds.
How did the coaches do?
So, how did the Cowboys' and Giants' coaching staffs perform in their respective decisions? Dallas faced fewer tough decisions (four shown in the table) and performed quite well, missing only one. That 1.5 percent GWC error for attempting a long field goal early in the game got a moderate confidence score of 5, meaning extreme factors (such as misevaluation of the teams' customized characteristics) could lead to a reversal of ZEUS' play-call choice.
Often, coaches get condemned by the media for "questionable" play calls after a loss. Sometimes, these criticisms are completely unfounded and based merely upon "playing results" or second-guessing. Unfortunately for the Giants' coaching staff, that wasn't the case Sunday. New York stumbled on six of the nine critical decisions analyzed here, and five of the six had confidences of 10. Let's take a closer look at these suboptimal decisions.
(1) Early in the second quarter with the score tied 7-7, the Giants encountered a fourth-and-7 from the Cowboys' 38. They chose to punt, which actually turned out well as they downed the ball on the Dallas 4-yard line. So how can we call this an error? Because in the long run, the team is better off trying for the first down when all possible outcomes are weighted properly. The actual result isn't relevant. Did the Giants know the ball would be downed on the 4-yard line? Did they know the ball wouldn't bounce into the end zone? Did they know that the Cowboys wouldn't run the punt back for a touchdown? All possible outcomes are evaluated by ZEUS, and each is given a weight based upon the probability of its outcome. Downing the ball inside the 5-yard-line (or even inside the 10-yard line) is an unlikely outcome. It just happened to be the result this time, but results don't determine what is right. As a simple example, consider a game of Russian roulette where the gun's chamber has one live bullet, one blank bullet, and four empty chambers. Suppose a player is told he will receive a $100 bill if the blank fires. He chooses to play, and it turns out to be his lucky day -- he holds the gun to his head, a "bang" is heard but his head remains unscathed. He actually fired the blank and receives $100 for his bravery! Do you think his decision to play this game was the right one?
(2) With time running out near the end of the half (1:30 remaining), the Giants had a fourth-and-1 at the Cowboys' 24-yard line. Instead of attempting a 42-yard field goal, New York tried for the first down and failed. Most coaches would have made a large (~3 percent) error by attempting the field goal. The Giants' minor error was attempting to run to the outside. Compiled for the entire NFL, running outside and running inside lead to similar results on average. The difference is that end runs gain more yardage when they work but lose more yardage when they don't. When only 1 yard is needed for a first down, it's easier to get it by running inside. Yes, you're more likely to get a TD by running outside, but ZEUS folds this in. It turns out here there is enough time to take the slow route and still have decent chances to score a TD, with a (later) field goal being the likely consolation prize. Instead of being blasted by the media for a bad decision (which is exactly what happened), the Giants should have been congratulated for not making the much bigger error of attempting a 42-yard field goal -- the second-guesser's (and most NFL coaches') play-call decision.
(3) Things just got worse for the Giants in the second half. Early in the third period, New York had a fourth-and-1 on its own 34, trailing by 3. We don't think there is a single NFL coach (and maybe not even any college coach) who would go for the first down here, but ZEUS says it's correct by 2.7 percent GWC. Converting on fourth-and-1 occurs more than half the time. Keeping a drive going is huge. Even though a failure gives the opponent great field position, it's still worth the risk, and it's not close.
(4 and 5) Twice within a 10-minute span the Giants ran into a fourth-and-goal situation from several yards out (5 and 4 respectively), both times trailing by 3 points. They were consistent, settling for the "sure" field goal to tie in each case, and ZEUS was consistent in dinging them for their errors. The second situation turns out to be twice as bad because the game is that much closer to the end. In either case a 4-point lead means the opponent needs more than a field goal to tie, and in either case a failure to convert the touchdown leaves the opponent with its back to the goal line. These two big factors combined swamp the "sure" 3 points (which are not guaranteed, by the way).
(6) The final Giants error is a surprising one. Having scored a TD with just over a minute remaining, the tying extra point is the automatic choice. Did any of the 70,000-plus Giants fans in attendance even consider that going for 2 could possibly be the right decision? Yet that's the right call, and again it's no contest! The explanation has to do with the fact that Dallas will be favored if the game goes into overtime. (If you believe the oddsmakers that the Cowboys were a 3:2 favorite to win the game from the opening kickoff, then they have an approximately 55 percent chance to win in overtime.) ZEUS says that by going for 2 points (and risking losing the game immediately), New York is still better off than simply settling for a tie and at best an extra period.
To see how this is possible requires a bit of math. Assign the symbol S to represent the probability that the Giants will stop the Cowboys from scoring in the remaining minute plus of regulation. Call T the chance that New York can succeed on a 2-point conversion. Then along that path, the Giants win by converting the 2 and stopping the Cowboys from a subsequent score. Their GWC by going for 2 points is T*S. If they settle for a tie score at this point, they must stop the Cowboys from scoring in the reminder of regulation (the same S from above) and then win in overtime (45 percent). Equating these two quantities tells us the break-even point needed to make the 2-point conversion worthwhile. T*S = S*45 percent or T=45 percent. Since the 2-point conversion rule was instituted, the NFL success rate is nearly 48 percent. New York has one of the best running games in the league, although that is partly offset by Dallas' well-above-average rushing defense. Still, if 48 percent is the historical NFL average and the Giants need only 45 percent chance to convert to make it right to go for 2, it is clear that putting the game on the line immediately is the right choice.
The team that made the fewest play-call errors beat the team that made the most play-call errors this time. And the team that lost got skewered by the media for its bad play calls, justifiably so this time. But it doesn't always turn out that way. For example, on NBC's nationally televised battle in Denver, Seattle prevailed in a similar back-and-forth game with a lot of late fireworks. With 3 minutes remaining and Denver down to one timeout, the Seahawks faced fourth-and-1 at the Broncos' 5-yard line, leading by 4. Instead of trying to drive the last nail in the coffin by going for it, they took the sure field goal and a 7-point lead. That decision was 1½ times worse than any single error made by the Giants -- more than 5 percent GWC. But because the Seahawks won the game, little attention was diverted from the celebration to assess the merits of their faulty play calling.
Chuck Bower is a Senior Scientist in Experimental Particle Astrophysics at Indiana University. Frank Frigo is a former backgammon world champion. You can e-mail them at firstname.lastname@example.org.