Bill James’ Total Runs
I love Bill James’ work. He is without a doubt one of the most influential people to have ever graced the game of Baseball. As the “father” of sabermetrics and the innovator of a plethora of different formulae that are still implemented (or have been improved on) to this day, I’d venture to say that he’s worthy of being inducted into the National Baseball Hall of Fame.
This isn’t to say that he’s infallible, though. He has certainly made his fair share of mistakes over the years with the manner in which he attempts to quantify player production. Win Shares, which is perhaps James’ gaudiest invention to date, fails to incorporate the concept of “Loss Shares” (which he’s finally beginning to develop), and there’s no baseline. Runs Created has gone through multiple adjustments over the years, to the point where it’s difficult to identify which one is which and which one is better (it’d be interesting to compare each rendition and check the RMSE to see if there’s been any actual improvement). James has deprecated the concept of linear weights in the past- I’m not sure if he’s softened his stance since, but it’s worth noting. Baseball and run scoring may not be linear; but it’s the only means with which we can actually quantify an individual hitter’s performance without his surrounding team affecting him.
Anyways, this is a long-winded way of saying that James has dropped the ball again, with another one of his new inventions. That’s not to say that he’s “lost it,” or that I’m smart- I can assure you, I’m not. He’s just made a mistake that’s fairly obvious (at least to someone that speaks amateur sabermetrics) and I’d like to point it out. Why, you may ask?
Because I believe he’s ignoring me. I sent Bill a message about his new statistic, Total Runs, on his official site- and I admit, it was worded poorly. Chalk that one up to being a bit starstruck. Hey, I idolize the guy, okay?
“Hi Bill, I have a question about “Total Runs.” Why not use Runs Created Above Average? Using straight Runs Created will rate a player with 100 RC and 400 outs equal to a player that has 100 RC and 450 outs as the same. Or am I completely overlooking something?
Asked by: J.T.
Answered: October 1, 2009
Probably. To be honest, I have no idea what you are talking about.”
Okay. So I wasn’t very clear. Not a problem. So I sent him another question explaining it a bit clearer, and he’s yet to reply. He’s responded to other questions and comments, just not mine. While I’d like to think that he’s taking my comment to heart and is fervently working away at fixing the error, his reputation for not listening to ideas about improving his formulae leads me to assume that he’s just choosing to ignore me. Then again, the last time I messaged him was asking whether or not he and John Dewan would convert their Plus/Minus defensive statistic into runs. He didn’t reply back then, and it was most likely because they were already in the process of doing so. My fingers are crossed that it’s the same case here, but I’m not so sure.
First of all, let me explain what “Total Runs” are (which I’ll denote as TR from here on out): TR “gives us one number to recognize the total contribution of each player in baseball,” as defined by James in The Fielding Bible, Vol. II (which is a definite must-read for any baseball fan). So there are three aspects of a position player’s game that he’s accounting for- hitting, baserunning, and defense.
All right. So Bill’s using his patented Runs Created to estimate runs contributed via hitting. I have my own biases against RC because it’s a team-affected run estimator, but to Bill’s credit, he does a lot to try and fix this issue by introducing “Theoretical Team Runs Created.” This is done with the intention of placing a hitter in a league average lineup to get an idea as to how he’d produce in a neutral hitting environment. Beautiful. It’s still not as good as theoretical team BaseRuns, but it’s not terrible either. At least it’s closer to linear weights. So, Bill can just use straight RC to evaluate hitting, right?
The short answer: no. This is the assumption that TR makes, and it is this assumption that makes the entire framework of TR flawed. Now, let me explain why this is by using an example.
Player A has 99 RC in 553 at-bats. Over the course of these 553 at-bats, he has also accumulated 409 outs (AB-H+CS+GIDP). Player B, on the other hand, has 99 RC in 525 at-bats and has accumulated 374 outs. This means that Player A has an R/O (runs divided by outs) of 0.24 and Player B has an R/O of 0.26. They’re obviously NOT the same hitter, but TR makes the assumption that they are.
Player A is Curtis Granderson and Player B is Andre Ethier. So if they’re not equal hitters like TR suggests, how do we go about fixing this issue?
It’s pretty simple. You create a baseline. There’s no need to mess around with the concept of “replacement levels” (or F.A.T., “freely available talent”). Since James is using baserunning runs above average and runs saved above average, let’s just stick with a baseline of “league average.” In order to do this, we first have to place them into a RPG (runs per game) context- how many runs a lineup of Grandersons and Ethiers would create.
Granderson is a 99/409*26.25 = 6.4 RPG player.
Ethier is a 99/374*26.25 = 6.9 RPG player.
For comparison’s sake, let’s assume that we’re playing in a 4.5 RPG environment. That’s our baseline. Runs Above Average is expressed as (RPG-LgRPG)/26.25*Outs.
Curtis Granderson: (6.4-4.5)/26.25*409 = 30 RAA
Andre Ethier: (6.9-4.5)/26.25*374 = 34 RAA
So that’s how you fix part of TR. What else is there to fix?
Well, James is using positional adjustments. Except they’re based on pure runs derived from the discrepancy between RC at a position and Runs Saved at the position. Since we’ve implemented a baseline into his offensive measurement, we have to implement a baseline into his positional adjustment as well. This is pretty easy and I won’t bore you with the details. This is how many runs each player is expected to save above or below a league average defender at any given position.
First Base: -13.8
Second Base: +4.1
Third Base: -3.3
Left Field: -8.8
Center Field: 1.1
Right Field: -7.3
Which just so happens to be nearly identical to the widely accepted positional values you see on FanGraphs based off of UZR studies, with the exception of third base. It’s so identical, in fact, that it has an “r” of 0.97.
Anyways, those are the changes that Bill James needs to make if he hopes to make this system valid. Otherwise it’s just a bunch of numbers that have absolutely no context whatsoever. Sure, it might give us a reasonable estimate of a player’s total contributions, but it’s not going to give us a precise one. I haven’t seen it catch on with anyone yet, and that’s a good sign- maybe he’ll be able to fix it before it becomes popular, if it ever does.
By the way, I recognize that I’m being extremely critical of James here. I’m not writing this to put him down- on the contrary, I’m writing this in the hopes that enough people will ask him to make these simple changes to the point where he feels like he needs to make a change.