Jason Bay and UZR
I began writing this piece not too long ago, but got sidetracked thanks to a plethora of work. Ah, the life of a student. While this piece is undoubtedly outdated, the “moral of the story” is still highly relevant- which is why I’ve decided to finish and post this.
Ever since UZR was modified, people have been pointing out the dramatic change in the ratings of certain players- most notably, Jason Bay. That makes sense, considering that there were a number of reports stating that teams were wary of handing Bay a large contract because of his poor defense. Some have taken this to mean that UZR can’t be trusted; it is inherently flawed and therefore must be dismissed or looked at with a heavy degree of skepticism.
I vehemently disagree. The numbers are being misinterpreted.
First and foremost, we’re looking at a very small sample of players that saw a drastic change in their UZR- in my study, only 1% of all eligible players from 2003-2009 (n = 941) saw a change of 10 runs or more in their UZR rating. 7.5% of all players saw a change of 5 runs or more. All in all, we’re looking at a very small change from the old UZR to the new. The overall r^2 is .906, and the mean average error is 2.1 runs. That’s it. Left field saw the largest change, with an r^2 of .782 and an MAE of 3.6. Given that the changes to UZR were primarily the handling of park factors, it comes to no surprise that LF has seen the largest difference. By my calculations, only a few players have seen a large change in their UZR- mostly players that reside in LF at Fenway Park or Wrigley Field. Does this change Jason Bay’s value?
Not as much as you might think. First, we must keep in mind that a left fielder makes about 320 putouts in a full season. That means we’re looking at a relatively small distribution of balls in play. As I pointed out before, if a player goes 93 for 320, he’s a .291 hitter. If he goes 86 for 320, he’s a .269 hitter. The difference between a good and an average hitter is a mere 7 hits. The difference between an average hitter and a .250 hitter? 6 hits. So if a player happens to make a few extra plays here and there that an average player doesn’t, then you’re looking at an artificial boost in talent. This is why we say it takes multiple years of defensive data to get an idea of a player’s “true” defensive talent. One full season in LF is about 46% of an offensive season, or about 2.2 full seasons afield.
That said, Jason Bay’s +1.9 UZR in 2009 means absolutely nothing about his defensive talent. In fact, if you use a 5/4/3/2 weighting with a simple aging adjustment, Bay’s estimated defensive talent level with the old UZR is around -12 runs per 150 games. The new UZR? -9. That’s a difference of three runs, which is right around the MAE of all LF. I understand the shock of this monstrous change, because it truly is unnerving. But we must keep in mind that it is nothing more than an outlier. Again, it appears that the media is making mountains out of molehills. Jason Bay is still a well below average defender by UZR, and one season- despite the dramatic change- does not mean much at all.
Another thing to keep in mind is that not all organizations rely on UZR to measure defense. The Red Sox’s proprietary system, which I believe was designed by Tom Tippett (one of the brightest saberists), indicated that Bay was a poor defensive outfielder. Other teams have come to the same conclusion, with their own metrics- so it’s not as if UZR determined the market for Bay. Scouts don’t think much of Bay, apparently, and neither do any of the defensive metrics. If it quacks like a duck, and waddles like a duck…well, you see where I’m going with this.
UZR is an imperfect metric due to measurement and recording errors, but it is an extremely useful tool. Use it wisely.