It’s no secret that MLB prospect success rates are rather low. There’s been a couple great studies on prospect success rates like Scott McKinney’s study on the success of BA’s top prospects and Matt Garrioch’s study on the draft’s success by round; however, I have not seen any studies on the success rates of draft picks by each individual slot. This has peaked my interest recently as the Cubs are tanking the 2012 season for as high of a draft pick as possible. Many of you already know I am working on a Ph.D. in History so something like this is right up my alley (too bad I chose a Civil Rights topic for my dissertation instead of a topic on baseball or I’d be done already). I decided to do my own study tracking the success rate of every draft pick in the first round from 1990-2006.
I separated the 17 years into three separate brackets. 1990-1995, 1996-2000, and 2000-2006. I chose to stop at the year 2006, because many of 2007′s prospects have their fates yet to be decided. The number 3 pick for instance, Josh Vitters, just hit the majors. Is it fair to call him a bust when he’s had 25 plate appearances and is still only 22 years old? Granted the more recent years will still be a little shady as well, especially players drafted out of high school, but at this point successful players should be performing at the majors.
Establishing what would be deemed a “successful” draft pick was the most difficult part of this study. I asked a few of the popular prospect experts how they would define success of an MLB draft pick, and the responses all came back similar; it depends on each individual case, the money involved, and where they were selected. There was no one way to define “success” that would cover every draft pick. So instead I chose three separate approaches.
First, I went with a similar methodology to Scott McKinney’s study on BA’s top 100 prospects. I used FanGraphs’ Wins Above Replacement (WAR) as the tool for measurement. I took the average of WAR at the MLB level during the player’s controllable years excluding seasons under 100 plate appearances or 25 innings pitched if they occurred in the first 1 or 2 consecutive seasons of reaching the majors. McKinney wrote that he “was attempting to account for the fact that many players get very little playing time in their first or second season, and I did not want to give them equal weight in the average WAR calculation. At the same time, I didn’t want to omit all short or partial seasons over a player’s cost controlled years because they are often due to injury or poor performance.” I agreed with this premise and decided to keep this stipulation. This is labeled as cWAR.
That means for a player to be deemed a “success,” they must post at least a 1.5 WAR average in the 6 years before they could hit free agency. In addition, there would be a “superior” category in which the player would need a 2.5 WAR average over that same time period.
Second, I looked at the peak of the player’s career. I evaluated the players’ best 5-year stretch using higher minimum WAR requirements than the first approach and labeled it as pWAR. Since I cherry picked the best 5 consecutive seasons of a player’s career, I increased the WAR to fit the basic definitions for a league average and superior player. That means the minimum success WAR was raised to 2.0, while the minimum superior WAR was raised to a 3.0 average. This would give “late-bloomers” a chance at being called a success, or some successes to be bumped up to superior players.
Lastly, I looked at the longevity of a career. To play in the majors for an extended period of time shows a player had enough skill to stick around at the highest level. I wanted a number high enough that the player received either a multiple year contract or multiple contracts after they hit free agency. I decided to use 10 years as the amount to be deemed a success for draft picks. There were 2 players in the next bracket who are both still active and sitting at 9 years, I decided these 2 players deserved the success as longevity; they are R.A. Dickey and Matt Thornton. After 1999 there were no players who were successful via only the longevity clause which was expected as there’s just not enough time to develop through the minors and reach 10 seasons yet. The same minimum requirements of 100 PAs or 25 innings pitched for the first year or two apply. I also decided to add a similar stipulation to the tail end of careers to eliminate failed comeback attempts to avoid artificially inflating a player’s length of career. Overall, a very small percentage of players were deemed a success via only longevity so I was very happy with how this approach turned out.
As you probably expected, the further the study went, the more fuzzy the picture became on some draft picks. These were guys, usually called up within the past few seasons, who sat on the fringe of success and could go either way depending on how the rest of their career goes. There wasn’t a significant amount of players affected to alter the results but there will definitely be an influx of more successful players as time goes by in addition to the handful of the successful by longevity guys as well. This type is actually rather easy to predict. Usually they are just on the border of being a good player. The majority were actually converted starting pitchers who went to the bullpen and found some success there after failing as a starter. Guys I would expect to see hit the 10 year mark include Phil Hughes, Chris Volstad, and Carlos Quentin among others. The study predicted players that I perceive as borderline very well. In all, I agreed with nearly every determination in the study. The few that I disagreed with, tended to have injuries skew their statistics.
One question I wanted definitively answered was if tanking actually helped teams rebuild quicker. I would answer that with a resounding yes after seeing how successful the higher draft picks were compared to the mid to low picks especially when separated into the brackets.
Teams that chose in the top half of the draft had at least a 36% to find a successful player moving forward. Teams choosing in the top 5 had nearly a 50/50 chance. While teams had a little better than a 1 in 5 chance to land a quality player in the 16-25 range, if you weren’t making the playoffs, you absolutely wanted to draft as high as possible.
As you can see the chance to find a superior player also drastically decreases the further you get in the draft. It was even more important for rebuilding teams to stay in the bottom 10 in the standings to have the best chance at acquiring impact talent. After the top 10, there is a significant drop off that levels off until the final bracket.
The second question I wanted to answer was if drafting had improved over the years as scouting and saber metrics had advanced. This is a little more difficult to answer but based purely on my research I would lean towards no. In the 90-95 bracket teams found a successful player 34% of the time and a superior player 18% of the time; however, those numbers dropped in the 96-00 bracket to 27% and 16%. In the final bracket, those numbers rebounded to 30% and 20%. While the successful player percentage is still lower than the first bracket, if you take into account that many players drafted between 00-06 have not been in the majors long enough to get out of their controlled years and there were zero longevity successes in this bracket, you could predict an increase of 12 more successes. That would increase the success percentage to 37% still within 3% of the 90-95′s success rate. With modern medical advances, nutrition awareness, and less general wear and tear on players, I think there’s a case to be made that we are seeing more successful players because injuries are less career threatening than ever before and players were able to keep a higher production and hang around the majors longer. You could also make the case that the 02′ and 05′ drafts are two of the best draft classes ever and those are skewing the numbers more positively. Moreover, there were a few cases like Brian Bogusevic who haven’t been in the majors very long, put up a really good season, and that one year that carried him to a success result. In any case, there’s no definitive evidence to conclude there was an improvement in scouting.
As for the Cubs, the organization wasn’t as bad as I expected. Out of 17 draft picks in the first round, they came away with 5 good picks and 2 superior players for just shy of a 30% success rate. The average success rate was 30.64%, and the Cubs were tied for 14th, right in the middle of the pack. However, there were two problems. Foremost, three of their picks’ careers were derailed by injury – Kerry Wood, Corey Patterson, and Mark Prior – and the team traded away Doug Glanville and Jon Garland for past their prime veterans. Second, there were zero successful picks after 2001. As Scouting Director from 1996-2002, Jim Hendry chose 4 of the 5 successful picks for a 57% success rate. The guy that assumed the role after Hendry, John Stockstill, went 0-3 before he left to join the Orioles. After Hendry turned to Tim Wilken, you can see a noticeable difference in talent through the draft with all four of his first picks already playing in the majors. Speaking of Tim Wilken, there’s a good reason he’s so highly regarded around the major leagues. While with Toronto and Tampa the two organizations went a combined 9-16 (56%). I am very happy the new front office kept him on board and have expanded his roles with the team.
I enjoyed working on this quite a bit. It was fun to take a stroll down memory lane, and it was also interesting to see what players I perceived as better or worse than they actually were. I had to triple check Corey Patterson’s numbers after they said he was a success… and he wasn’t the only player/team I was surprised by.
I’m not done with this study. I’m already expanding to the team stats and I will definitely revisit the study in a few years to update and expand it for more recent drafts. I will probably go back earlier than 1990 as well to continue to investigate if the success rate of organizations has improved or diminished over the decades.
If you have any suggestions how to improve the study, found an error or want to share any surprises you found, please leave them in the comments.