Zone Rating Runs, 2010
As a follower of sabermetrics, I am very much in favor of progressive metrics designed to better value a player’s contributions on the field. Certain metrics created over the years have fallen out in favor of newer, more accurate ones. Runs Created, for example, had a nice run- but it has since been supplanted by David Smyth’s BaseRuns and, when it comes to individual hitters, Linear Weights. The same holds true for fielding metrics. We first began with fielding percentage, which was improved upon by range factor, which turned into fielding runs, which turned into Zone Rating, which turned into both Ultimate Zone Rating (UZR) and Defensive Runs Saved (DRS). The difference and usefulness of using BsR over RC is pretty straightforward- one obvious reason is that BsR counts all home runs as exactly one run; RC doesn’t. It gets a lot more confusing with defense, though, because we’re not really sure what we’re dealing with.
Basic ZR counts the number of balls hit into and around a fielder’s “zone of responsibility,” tracks the plays made, and divides plays by zone chances to estimate the player’s defensive efficiency. UZR and DRS take this simple framework and make a multitude of adjustments, accounting for things like batted ball speed, the base/out situation, and using smaller sub-zones in order to get a better idea of just how efficient the player was. Rather than underrating players with exceptional range (because we’re counting both balls in zone and out of zone plays made as zone chances), we’re getting a much more accurate portrayal of the player’s value provided on the field because of the inclusion of so many more variables.
But this might not be true. Chances are pretty good that we’re really not getting a better idea of a player’s fielding ability; chances are we’re introducing a bunch of noise. Think of it this way: if a player is standing on the edge of his zone and takes one step over to make a play considered to be outside of his zone, he gets credit for an out of zone play. Except…all he had to do was move a foot over. That’s not flashing exceptional range; it’s receiving false credit. The difference between “in zone” and “out of zone” is difficult to ascertain; this becomes infinitely more problematic when we break down the zones into smaller ones and add things like batted ball speed. That’s not the only problem, though- there’s a rather large discrepancy between data providers that supposedly track the same thing.
ZR was originally provided by STATS, Inc. before also being provided by Baseball Info Solutions (BIS). The major difference between the two, on the surface, is that BIS ZR excludes out of zone plays from the numerator and the denominator; leaving us with both defensive efficiency within the zone, along with a number of out of zone chances he made. This is theoretically more precise, but it is far from practical due to the aforementioned issues with the precise location of the batted balls. There’s another difference, which makes matters infinitely worse: the two are sometimes miles apart on players. Much has been made of this when it comes to UZR- you can see some of the differences highlighted in one of Tango’s threads, in which we discover that Andruw Jones was rated +112 runs by BIS UZR (bUZR) and -5 by STATS UZR (sUZR) from 2003-2008. That’s a remarkable difference. Carlos Beltran showed 77 runs of difference, and Adam Everett 51 runs. This is no small difference- remember that approximately ten runs equals a win, and you’re seeing 5-10 win differences over a six-year period. Defensive metrics cannot be considered reliable when two major information providers, STATS and BIS, give such wildly different outcomes.
I wrote about the state of defensive metrics a while back and stated that I favor ZR over UZR or DRS due to the uncertainty in the metrics. ZR, by using one large, simple zone, should help minimize the effects of bias- at least, by a little bit. And to help smooth out the differences between providers, I’ve decided to average the two in order to get a more complete estimate of the player’s defensive efficiency. Just by looking at the 2010 data, there’s some serious discrepancies that worry me- the average error of estimated chances of qualified players (900+ innings afield) is about 22; the average error of plays made (which should be reasonably easy to determine) is 21, and the average error of runs saved or cost is five runs. And this is comparing apples to apples with the simplest comparison imaginable. It’s no wonder that sUZR and bUZR are so remarkably different.
The players that sZR and bZR disagreed on the most in 2010: Hunter Pence (15 runs), Adrian Belte (14), and Gaby Sanchez, Robinson Cano, and Marco Scutaro (13 runs). In all, there are eleven players in the sample- approximately 9%- that have a difference of ten runs or greater, and another twenty that have a difference of 7-9 runs. The systems will most likely show varying levels of agreement/disagreement based on position, as well. This is something I intend on looking deeper into at some point in the future.
The all ZR team for 2010:
1B: Daric Barton, +10
2B: Mark Ellis and Ian Kinsler, +10
3B: Jose Lopez, +16
SS: Brendan Ryan, +17
LF: Juan Pierre, +7
CF: Denard Span, +6
RF: Jay Bruce, +12
You can find the spreadsheet containing data for all players in 2010, including their STATS and BIS chances and plays made, here.