Who would you rather have a discussion in proportionalityFebruary 16th, 2013 by terrybenish
Clubhouse Confidential is a thirty minute long, five day a week, show hosted by Brian Kenny on MLB the network. Kenny the former ESPN talking head that did a lot of boxing and baseball work there is great. The show which I really enjoy and fully recommend attempts to marshal all Sabermetric tools to evaluate and compare players. Last night and this morning they ran a piece contrasting Torii Hunter with Josh Hamilton. Running a disclaimer against both age differences and mentioning how defensive metrics are flawed, they contrasted the two player’s Wins Above Replacement (WAR) numbers. As I’ve mentioned in passing and referenced troubles in the defensive aspect of WAR, which is Ultimate Zone Rating (UZR), as being badly flawed, they plunge ahead and throw in the defensive stat of Runs Saved. That stat, as is UZR, is a proprietary number produced by a company that purports to study film of every play made by every player and then using a formula calculate the net number of runs saved. The tool can be negative. For a given player it can range to be wildly positive and wildly negative. Below I have the link to Baseball Reference’s pages for both Hamilton and Hunter.
The other point or failure here is that they did not play the same position last year. Hamilton split time between center field and left field, while Hunter played right field. An observable stat, unlike UZR or Runs Saved, is range factor. No gimmicks, no black box. Using that stat to see where they stacked up last year is valuable if not perfect in so far as they played different positions, we can see how they ranked at those positions. Hunter in terms of his range factor was the fifth best right fielder in the American League last year. While Hamilton if he qualified in terms of number of games in cf would have been eighth best and in left field with the same disclaimer, seventh. In terms of assists and double plays there is not much to separate them.
Using recognizable stats (range factor) there is not a clear advantage to Hunter over Hamilton at all. While on offense Hamilton dwarfs Hunter. Over the last two or three years the differential in OPS being one hundred to three hundred points on OPS. While Hamilton has missed games, he still generates more runs than Hunter. Not being able to pry into the black box stuff, the purported defensive advantage to Hunter is enough to even the value between them. If the formula in WAR is 50-50, offense to defense, which allegedly it is 75-25 offense to defense then the sway of UZR is enormous and probably very misleading at best. In other words the numbers are probably questionable, but the proportionality seems askew.
Good analysis relies on good analysts. Not a simplistic reliance on reliable analytical tools. It should certainly not rest on demonstrably flawed tools.
It is work like this that stalls out the use of Sabermetrics in general.