Evaluating Our Free Agent Contract Predictions
Introduction:
As the baseball season approaches, it's time to review and analyze our free agent contract predictions. This year, I'm focusing on my own predictions and those of the crowd, aiming to assess our accuracy and identify areas for improvement. The article highlights the key findings and provides a comprehensive evaluation of our performance.
Methodology:
To evaluate our predictions, I categorized the signings into three groups: hitters, starting pitchers, and relievers. I also examined the entire Top 50 without positional separation. The primary metric for accuracy was a formula discussed earlier this winter, but I also considered average annual value, total guarantee, and number of years. The analysis aimed to determine who performed best in predicting the broad market and who predicted individual free agents most accurately.
Results:
Too Hot:
- Munetaka Murakami: Overestimated by $90M.
- Kyle Tucker: Overestimated by $37M.
- Tatsuya Imai: Overestimated by $26M.
- Eugenio Suárez: Overestimated by $25M.
- Josh Naylor: Overestimated by $24M.
The analysis reveals that the crowd didn't escape Murakami's madness, but great work was done on the rest. The misses were attributed to pillow contract players and late signings.
Too Cold:
- Michael King: Underestimated by $29M.
- Cody Bellinger: Underestimated by $23M.
- Devin Williams: Underestimated by $19M.
- Pete Alonso: Underestimated by $15M.
- Alex Bregman: Underestimated by $12M.
The crowd's misses were attributed to slightly higher demand for good hitters, with no equivalent to the NPB problem. The analysis suggests that the misses are spread across hitters, starters, and relievers.
Just Right:
- Ranger Suárez: Predicted correctly.
- Ryan O'Hearn: Predicted correctly.
- Brad Keller: Predicted correctly.
- Dylan Cease: Predicted correctly.
- Robert Suárez: Predicted correctly.
The crowd's predictions were also close, with Edwin Díaz, Josh Naylor, Shota Imanaga, Kazuma Okamoto, and Tatsuya Imai's deals predicted within $1M.
Overall Market and Final Thoughts:
The market for pitchers ran hot early and trailed off, while the market for hitters never really got going. The crowd's predictions were the only ones that didn't overshoot, and they were the best in some order when compared to other prognosticators. The analysis concludes that the crowd did an excellent job, and their predictions are preferred over professional prognosticators in most years.
Appendix:
The appendix includes tables showing error metrics for various ways of scoring predictions, including year-adjustment methods, AAV forecasting error, total guarantee forecasting error, and a non-monetary comparison of predicted and actual years for each contract.
