February 27, 2004
Birdhouse Interview Series with Jeff Luhnow
Part Three: Systems and the Advisory Council
By Brian Walton
Here, in the third of four installments of my interview with Jeff Luhnow, the Cardinals Vice President for Baseball Development talks about the systems his group are developing and how they plan to deploy them.
When this series is complete, you should have a lot better idea about how the team off the field has clear plans on how to improve the team on the field in the future. And it’s a lot more than just sabermetrics.
Part One: The Job and Fantasy Baseball
Part Two: Success, Run Like a Business and Relationships
Today: Systems and the Advisory Council
Part Four: Player Evaluation
BW: Sticking with tools, some teams, like Cleveland with DiamondView, developed in-house proprietary systems, while others have bought them. What is the Cardinals’ approach?
JL: Cleveland’s system has been written about a lot. From what I’ve read about it, it’s a great system because it incorporates a lot of different aspects of evaluating players. From the scouting standpoint, we recently signed up with a company called E Solutions and it’s a piece of software that several other teams are using.
BW: ScoutAdvisor?
JL: Right, ScoutAdvisor. It’s a good tool for the scouts to input all their information after they’ve gone out and seen players. It’s a good database to go through and search and find reports; a good way to organize the data. But what it doesn’t do is really give you any insight beyond what is in the report that the scout filled in. So, we’re going to be building databases internally and building analytical tools that go on top of those databases to really filter through the data and figure out how we can evaluate players relative to one another. Now, in this discussion today, we heard a lot about context. A player might have a great statistics one year and not so great statistics the next year. It could be that is normal statistical fluctuation based off their performance level. It also could be that their context changed. They went from one field to another; from one environment to another; they played on a different team; they played against different pitchers, etc. As someone said in there, there are so many variables that it’s hard to judge just the batter himself because there were so many other variables that contributed to his performance.
(Note: This conversation was held in conjunction with Ron Shandler’s First Pitch Forums, which are being held this weekend in Baltimore/Washington, New York and Boston. You can still enroll here.)
BW: Many of those variables which aren’t under the batter’s control?
JL: Yes. However, if you’re smart about the way you do the analysis, you can begin to isolate those variables and neutralize them to a certain extent. And put all the players back on a level playing field. Once you do that, at least it’s a more level playing field to compare one player to another. We all know the example of Colorado versus Dodger Stadium, but there are so many other factors that go into it, like quality of the pitching that a batter faced, things like that that you can actually quantify to a certain extent because all the statistics for the pitchers are available. You can begin to take that out of the data and view everyone through the same lens. And I still think that you still run the risk that you’re going to evaluate one player better than he should be; but, less of a risk. That is really what it is all about, is minimizing the risk, so that we have at the time we are making the decisions the best available information. It’s not just the best available statistics, but the way we filter, massage and manipulate those statistics to gain some insights.
BW: E Solutions is a host-based Lotus Domino system. Are you also going to run in-house systems?
JL: Sure. And one of the reasons we hired Mark (Johnson – see Part One), not only does he have a fantastic mathematical background; he has been coding software for the past four or five years for Yahoo out in California. So, he is very adept at database development and also front-end software development. And, we already have started doing a lot of that stuff. We’ve got some websites that are just going to be internal websites to share information among scouts, for example, and then programs that we’re going to put on top of the database that analyze and evaluate the statistics.
BW: Will the Advisory Council have access to your internal databases or will you be talking with them more from their terms coming in?
JL: It is not decided yet. Tomorrow, we kick off our relationship more formally. (Note: The Cardinals and the Baseball HQ team met on Monday, February 23.) Now that everything has been signed, they are going to come down to Busch Stadium. Two will be there in person and two on the phone and we’re going to talk about the plan going forward. We are going to be relying on the Advisory Board to give us unique perspectives, to give us their points of view about a lot of things that we need to make decisions about. I suspect that they will have influence and access to the tools that we are developing internally. We have confidentiality agreements with them. I think we are going to need to share to get the maximum amount out of them. So, if they’re telling us one thing and we see it completely differently, it makes sense to have that dialogue.
BW: You have to be able to share the data and compare how each of you came to your views…
JL: Exactly. For example, I noted one earlier, Aaron Rowand. There were some comments made in there that Rowand may not being quite the player that some people think he is. Some very preliminary stuff that we had, showed him as a fantastic player. So, one of the things we are going to do is to get into that. Why does our stuff show him as having a huge upside?
BW: After all, he could become available this spring, and if so, you might have to make a decision as to whether or not to pursue him…
JL: Exactly. That is the sort of stuff that is worth debating. At the end of the day, similar to the professional environment that I grew up in as a consultant at McKinsey, the dialogue, the debate, usually leads to the best answer. Unless you foster an environment where people with a dissenting opinion not only have the opportunity to express it, but have the obligation to express it, you don’t end up with the best answer; the best result. That is the type of environment that we are trying to create within the Cardinals by bringing in outsiders, but also signing them up to confidentiality agreements so you can have open dialogue about the issues and debate the different sides and ultimately lead us to the right answer.
BW: You mentioned one area, defense, which traditional sabermetric study does not take as deeply into account. Have you made a decision yet on who will cover this on the Advisory Board?
JL: We are in the process of that right now. There have been a number of people. I know it is one of the areas that Bill James has been working on, but he is working on it I think exclusively for the Red Sox. We have our plans and I can’t really expand beyond that, but we do have our plans. We do consider defense to be one of the leading-edge frontier areas, if you will, of sabermetrics. It is incredibly important in evaluating a player.
BW: Can you give an example?
JL: You could have two players that create twenty runs, if you use the runs created analogy, but if one of them saves ten runs, and the other one doesn’t save any runs on the defensive side, you’d evaluate those players very differently. In traditional rotisserie, you might give them the same rating and pay the same amount for them. But in the way that we need to think about it, that’s ten runs that matter for us. A run saved on defense is equivalent to a run created on offense, from our point of view. We need to have the right tools to evaluate defense along with offense.
BW: And pitching?
JL: The same thing goes for pitching. It’s really the combination of the three. Bill James had a big breakthrough with win shares, in that it is one metric that does look at pitching, fielding and hitting. We’re thinking about our own tools and our own method and our own approaches and so forth. But, it’s important that we learn from that, that it’s important to have one metric that combines the three. Because when you are making a decision, it really is about, at the end of the day, how many games you win and lose. And there’s a tie between that and how many runs you score or allow or give up. That’s how we’re looking at it.
