Nothing brings out the baser nature of people quite like talking about baseball players' salaries: fans love to complain that players are pretty much slackers or spoiled brats, getting paid to do something that they'd do for free.
Nowhere is this manifested more starkly than when we talk about a player's "walk year." In sports bars, on buses, on talk radio, in heated discussions at fantasy auctions, and even in reputable publications, you hear "he'll have a good year this year, because he's a free agent after the season."
The accusation, of course, is that the player doesn't usually try his hardest because he's not incented by an otherworldly payout at the end of the season. This year, with walk-year players like Gary Sheffield, Javy Lopez and Livan Hernandez all having unexpectedly good seasons, the debates are certain to be fast and furious.
Is there any truth to it? Do players perform better during the season immediately before free agency? Player performance varies quite a bit from season to season anyway: guys get dinged up, the schedule doesn't even out, a player's role shifts. And all of us (with the possible exceptions of Barry Bonds and Edgar Martinez) suffer the effects of advancing years.
Let's take a look at the players who will be free agents after the 2003 season, and examine their performance relative to what one could realistically expect, and see whether or not there really is a difference.
For a comparison baseline, we'll use Baseball Prospectus' PECOTA forecasting system.
PECOTA uses recent performance records, comparable player histories, the probable effects of aging, and the characteristics of a player's home park to deduce a set of predictions. Instead of just spitting out a single set of numbers, PECOTA provides a range of likely performances. For example, one guy who could have been a free agent this offseason, but signed a new deal with his club is Kirk Rueter of the Giants.
Here are Rueter's stats thus far, and the 50th percentile of his PECOTA forecast for 2003:
Using ERA as the metric for a starting pitcher's overall effectiveness, Rueter has performed below the 50th percentile of his forecast for 2003. That is to say, he's done slightly worse than expected; Rueter's performance is closer to his 40th percentile forecast (4.83 ERA).
We can look at the cohort of players eligible for free agency after the 2003 season, and compare their performance to similar players who are signed to guaranteed contracts for 2004. If there is a "motivation effect" that causes players to perform better in their free agency season, we should see the Free Agent group perform better than the non-Free Agent (control) group, relative to their respective projections. (In order to focus on this motivation effect, I've eliminated players for whom the club holds an option for the 2004 season, who fall somewhere in between the two extremes).
There are 78 hitters who have accumulated 150 plate appearances or more thus far this season who will be eligible for free agency after the year is out, and 105 hitters meeting the same playing time requirement who are between 29 and 34 but will not be eligible for free agency. Comparing each group's actual performance against the median of their respective PECOTA ranges produces some interesting results:
At first glance, there does appear to be something to the motivation effect. The non-free agent hitters are split almost evenly down the middle between overperformance and underperformance, just as you'd expect. But a higher fraction of those hitters eligible for free agency -- nearly 60 percent -- are outperforming their median expectation. The difference isn't huge, but it is noteworthy.
What about pitchers? There are 45 pitchers with 40 innings pitched or more who will be eligible for free agency after the 2003 season, and 74 pitchers between the ages of 29 and 34 who will not be eligible for free agency. The same examination of their performance shows the following:
No real surprises there. For the statistically inclined, the difference between the groups of hitters is statistically significant at the 90 percent level, and the difference between the groups of pitchers is not statistically significant at all. That might be indicative of a real effect, with hitters being better able to take advantage of offseason preparation and training when there's money on the line. Is it possible that the fan bleating about salary drive years is accurate?
It's possible, but there are limitations to the conclusions one can reasonably draw from this cursory study. First off, our metrics aren't perfect. For example, using ERA for pitchers is only an approximation. Brian Anderson of the Royals has a 3.88 ERA, placing him in the 90th+ percentile of his expected performance for the year. But in addition to those 3.88 earned runs per game, he's allowed 1.51 unearned runs per game, for a grand total of 5.39 runs per game. OK, so Anderson's been on the mound in front of defenses normally perpetrated by teams proudly bearing "Chico's Bail Bonds" on the back, but he's also allowed 174 hits and 23 home runs in 160 innings. So there are flaws in the measurements we're using, both in terms of inclusion and exclusion.
Furthermore, the control group we used here isn't perfect; players between the ages of 29 and 34 were chosen primarily for ease, rather than perfect accuracy. Add in that we're only looking at one year's worth of data, and it's hard to put a lot of weight on this quick statistical glance.
But this quick statistical glance does indicate that there might well be something to the motivational effect of the walk year, at least for hitters. After all, doesn't everyone work a little harder -- or at least make it look like they're working harder -- when the boss is figuring out bonuses for the year? And is it too early to talk about Nick Johnson for MVP in 2007?
You can check out more work from the team of writers of the Baseball Prospectus at baseballprospectus.com.