Anyone who plays in a Yahoo Fantasy Football league understands the frustration of relying on Yahoo’s fantasy forecasts. The projected stats always seem to be wrong. To quantify exactly how bad Yahoo is at projecting future performance, we’re going to compare Yahoo’s projected 2021 quarterback statistics to the naïve forecast of 2020 statistics. Our hypothesis is that the 2020 quarterback stats are a better predictor of 2021 quarterback performance than the Yahoo 2021 projections.
To compare the accuracy of Yahoo Fantasy projections for the 2021 season versus the naïve forecast (the 2020 Football season), three measurements are used:
The 2021 season was 17 games while 2020 was only 16 games, so the 2020 numbers are adjusted by taking a per-game average and applying it over 17 games (diving the season totals by 16 and then multiplying by 17.)
The Yahoo projections are from August 3, 2021, and the season totals are both from NFL.com. Because Yahoo’s projected points vary according to each fantasy league’s rules, only 2021 projections of the following statistics will be used:
- Passing Yards
- Touchdowns
- Interceptions
ALL QBs WHO PLAYED
Limiting comparisons to all quarterbacks with Yahoo projections who played in both 2021 and 2020, leaves 57 NFL quarterbacks to compare.
Note that these criteria exclude all 2021 rookies and quarterbacks who didn’t play in 2020:
Trevor Lawrence, Justin Fields, Trey Lance, Zach Wilson, Jordan Love, Davis Mills, Ian Book, Jake Fromm, Jacob Eason, Feleipe Franks, Kellen Mond, Mac Jones, Josh Johnson, Trevor Siemian, Sean Mannion, Cooper Rush, Mike White, and Josh Rosen.
Also excluded is DeShaun Watson, who played in 2020 and received projections from Yahoo for 2021, but didn’t play in a single 2021 season game due to “non-injury reasons/personal matter.”
Forecasting errors are the difference between the predicted value and actual value. Yahoo projects that Tom Brady is going to pass for 5,257 yards in 2021 but he ends up throwing only 5,316. The error is 5,257 – 5,316 = -59.
Mean absolute error (MAE) is the simple average of the absolute value for each error. Yahoo’s absolute error for Brady is |-59| = 59.
Mean squared error (MSE) squares each error, and then takes a simple average of all the squared errors. By squaring the error, MSE penalizes larger errors more than smaller errors. Yahoo’s squared error for Brady is -592 = 3,481.
For both MAE and MSE, lower is better. Using those two metrics, Yahoo outperformed the naïve forecast in our three categories of interest:
| Metric | Statistic | Yahoo | Naïve |
| MAE | Passing Yards | 711.42 | 1,951.96 |
| MAE | Touchdowns | 5.54 | 7.00 |
| MAE | Interceptions | 3.11 | 4.17 |
| MSE | Passing Yards | 1,179,872.68 | 6,988,153.87 |
| MSE | Touchdowns | 67.26 | 86.77 |
| MSE | Interceptions | 26.16 | 29.51 |
ALL QBs WITH 200+ PASS ATTEMPTS
Not all 57 Quarterbacks saw significant playing time, let alone any playing time. It doesn’t make sense to reward a forecast for correctly predicting David Blough’s 0 passing yards, 0 touchdowns, and 0 interceptions in 2021. Some quarterbacks never really had a chance of receiving significant playing time and others were injured, which is difficult to predict. (Except for Jimmy Garoppolo. Unfortunately, he will be injured this upcoming season.)
Limiting comparisons to the 24 quarterbacks who had at least 200 attempts in both the 2020 and 2021 seasons, Yahoo again outperforms the naïve forecast:
| Metric | Statistic | Yahoo | Naïve |
| MAE | Passing Yards | 694.92 | 3,717.95 |
| MAE | Touchdowns | 6.98 | 8.50 |
| MAE | Interceptions | 3.96 | 4.38 |
| MSE | Passing Yards | 757,169.00 | 1,471,3345.16 |
| MSE | Touchdowns | 74.73 | 117.08 |
| MSE | Interceptions | 23.42 | 25.69 |
ACCURACY OF QB RANKINGS
Excluding certain dynasty fantasy leagues where teams have some level of continuity year-to-year, the rankings of players is arguably more important than accurately estimating individual statistics. It matters less about how accurate are forecasts of Patrick Mahomes’s passing yards and touchdowns than whether Patrick Mahomes will generate more points than other available quarterbacks in a particular fantasy league’s draft.
Spearman’s rank correlation coefficient is used below to measure the correlation between the rankings of each forecast with the 2021 actuals. Spearman’s coefficient ranges from -1 to 1, where 1 is perfect correlation, 0 means no correlation, and -1 is a perfect inverse correlation. The closer to 1, the better the correlation between rankings.
If you are not a statistician and do not desire to become a statistician, then just think of p-values as the probability of the correlation occurring due to chance. A p-value under 0.05 means the results are typically considered statistically significant.
Comparing the rank correlation of both forecasts using the 57 quarterbacks who played in both 2020 and 2021, we see Yahoo outperforming the naïve forecast:
| Forecast | Statistic | Coefficient | p-value |
| Yahoo | Passing Yards | 0.83 | 1.19e-15 |
| Naïve | Passing Yards | 0.69 | 2.14e-09 |
| Yahoo | Touchdowns | 0.85 | 7.52e-17 |
| Naïve | Touchdowns | 0.67 | 9.99e-09 |
| Yahoo | Interceptions | 0.65 | 5.46e-08 |
| Naïve | Interceptions | 0.46 | 3.26e-04 |
Among the 24 quarterbacks with 200+ passing attempts in both 2020 and 2021, Yahoo wins again:
| Forecast | Statistic | Coefficient | p-value |
| Yahoo | Passing Yards | 0.72 | 5.7e-05 |
| Naïve | Passing Yards | 0.65 | 5.9e-04 |
| Yahoo | Touchdowns | 0.81 | 1.0e-06 |
| Naïve | Touchdowns | 0.55 | 5.7e-03 |
| Yahoo | Interceptions | -0.09 | 0.66 |
| Naïve | Interceptions | -0.32 | 0.13 |
Please note that the negative correlation among interceptions is suspect because of the high p-values.
CONCLUSION
After comparing the predictive ability of the 2020 season data versus Yahoo Fantasy Football projections, it looks like Yahoo was a better at predicting quarterback performance, based on passing yards, touchdowns, and interceptions.
There are many qualifications about the above analysis that will hopefully be looked at in the future:
- Only quarterbacks were compared.
- Only one season of predictions was used.
- While fantasy football is typically points-based, only three football statistics were used.
- Only three of many metrics were used to measure each forecast.
- The actual number of games played by a quarterback was not factored in adjusting 2020 season totals from 16 to 17 games.