The now not-so-new FiveThirtyEight.com came to life with all sorts of buzz … especially for those of us in the numbers geek world. It even coined a meta-phrase for a concept that's been around for a while … Data Journalism! Hey, we do data journalism here! I mean we run CBOE Volatility Index (VIX) data all the time and it turns into something resembling journalism, right?
Dirty little secret, though. You can often use data to tell the story you want to tell. You start with a thesis, say "high VIX leads to X outcome in a month," run the data, and if the data you run supports your thesis, great! If not, well, run different filters -- or change your thesis. Of course, if there was one perfect VIX rule, everyone would know it and use it and … well, it would stop working at that point anyway.
Which brings me back to FiveThirtyEight.com, and real fun with numbers. Take this post from the other day, "Billion-Dollar Billy Beane."
It basically starts with a conclusion: The Red Sox had interest in Billy Beane as their general manager (GM) back about 12 years ago, but the deal never happened … and looking backwards, he could have provided $1 billion or so of value to them. The piece supports this conclusion with well-written prose and lots of fancy numbers, and on the surface it's a fascinating read. Then, you think about for a few minutes, and ... well, it doesn't bear an awful lot of resemblance to reality.
So, in that spirit, let me lead off with my conclusion: There's a good case that Billy Beane has added more value than any GM in Major League Baseball (MLB) history, but he's not anywhere close to the $1 billion or so they come up with, or the $300 million they maybe should have paid him. Certainly not to the Red Sox … I could make a way better case he was worth that to a middle- or low-market team.
That's because the goals of a big-market team and a small-market team are not identical. I'm not saying Beane couldn't translate the skills of running the A's to running a team with big resources, I'm sure he could. It's just that you can't simply superimpose how he structured the A's and their payroll, add money, and spit out reliable hypothetical results.
The author goes through many machinations to determine the cost of a "win" in MLB over the last 15 seasons. He then translates this into how many wins you should get per dollar spent on payroll, and determines how much excess value Billy Beane has produced, based on how many wins the A's have versus how many wins their payroll predicted.
Spoiler Alert: He produced quite a few extra wins. But, it assumes a linearity in wins that has no basis in reality.
In VIX world, I could (and have) run data to find the correlation between VIX moves and SPDR S&P 500 ETF Trust (SPY) moves, then come up with an equation that best relates the two. The "best fit" is a multiplier of roughly negative six … that is, take the percent move in SPY, multiply by negative six, and you get an approximate prediction for what the VIX "should" move. Except it's not really linear -- bigger SPY moves will lead to exponentially larger VIX moves. Running a logarithmic estimate would yield much "better" results for outliers, but would have bigger errors on more normal days.
The biggest error in the Beane piece, in my honest opinion, is completely omitting any knowledge of the concept of marginal wins. In MLB terms, you wouldn't pay much money to improve your team from 70 wins to 71 wins, but you might pay a lot to go from 90 to 91. It leaves us with this:
In the last 12 years, the Red Sox spent $1.714 billion on payroll, while the A's spent $736 million. We can then break down what it could have looked like if Beane had worked for the Red Sox like so:
- Let's say it would have cost Boston the same $736 million that it cost Oakland to get the A's performance with Beane.
- At the hypothetical $25 million-per-year salary I suggested earlier, Beane would have cost the Red Sox another $300 million. (It's possible that Beane would have wanted more, but it's even more possible that they could have gotten him for less.)
- The difference in performance between the A's and the Red Sox over that period (where the Sox were as successful as at any point in the franchise's history, and the A's were supposedly stagnating after Beane's early success) has been about 50 games for Boston. Since we don't know exactly how good Beane would be at procuring additional wins above his Oakland performance, let's assume that the Red Sox would have had to pay the typical amount teams have paid for wins in the period to make up the difference. According to the year-by-year price of wins from my calculations above, those 50 wins (taking when they happened into account) would have a market value of about $370 million (though this might have been lower with Beane in charge).
Those extra "about 50 wins" Boston achieved were playing in the AL East, which for most of the last 12 years was a tougher division than the AL West, where the A's play. And I believe it was 56 wins anyway. So, let's say when we adjust for strength of schedule, we conservatively add four to 10 wins, and say there's a 60 win-gap between the teams, or five wins per season.
But importantly, all wins are not created equal. Going into a season, if a team has about a 70-win talent, it would not make any financial sense to spend the $37 million his numbers above imply that it would take to improve to a 75-win team. That's because you're not making the playoffs anyway, or even sniffing the playoffs and driving up attendance and TV viewership. And in reality, all teams in that part of the curve not named "Phillies" sacrifice wins today for more wins in a future season. It's a relatively easy and inexpensive part of the curve to show improvement.
On the other hand, if you have an 85-win talent, it's probably worth considerably more than $37 million to bump it up to a 90-win talent. It's quite possibly the difference between sitting at home and making the playoffs. Ergo, the value of a marginal win for a team at this end of the curve is considerably larger than his numbers suggest.
If hypothetical Billy Beane came to Boston, perhaps he could build an 85-win-caliber team on his same A's payrolls. But to get to the 90-win-or-so level of the typical, actual Red Sox, he'd cumulatively spend way more than the $370 million cited above. He would both have the incentive (the hyper-inflated value to him of marginal wins in that part of the curve) and the financial means to do so. And this all sets aside the fact that hypothetical Billy Beane would lose his job if he turned a 90 win-ish Red Sox team into a financially efficient 85-win team.
But, back to the real world. You know who just paid up big time for marginal wins at the high end of the curve? Actual Billy Beane.
The A's recently traded their best prospect, Addison Russell, to the Cubs for two pitchers that can help them greatly in the here and now, Jeff Samardzija and Jason Hammel. The current A's pick up approximately two wins worth of value in the current season. The future A's give up maybe something like $50 million in value, a ballpark guess of the underpricing of Russell's potential production versus what the Cubs will have to pay him over the next five years or so. Hammel is a free agent after this year and Samardzija goes to arbitration, so money associated with them is pretty much a wash. In other words, based on their salaries this year, they're close enough to fairly priced that it's not really a factor in the trade.
These are all approximate estimates, so long story short, the A's paid maybe $20-$30 million in present value per 2014 "win." Depending on how it plays out, that might become a bargain if it's the difference between winning their division instead of a wild card and/or winning the World Series thanks to their bulked up rotation.
My long-winded point is that those extra 60-or-so adjusted Red Sox wins are worth way more than $370 million, given the part of the win curve where both teams resided. What's more, the whole win curve isn't linear to begin with. If the average team can pay "X" for a win, a team with greater revenue can pay way more than X. And, if they're an above-average team, they will most certainly pay way more than X, as their added revenue is the advantage they can and should leverage the most.
So, actual Billy Beane running the actual Red Sox would run the team based on his actual budget. It's entirely possible he would have figured out a way to reduce payroll or increase production, but nowhere near to the extent the FiveThirtyEight piece suggests.
Disclaimer: Mr. Warner's opinions expressed above do not necessarily represent the views of Schaeffer's Investment Research.