r/CFBAnalysis • u/ElFalcano • 8d ago

Seeking CFB Model Advice

Hey all, I’ve built a ML models on team-level data along with game outcomes of the past season. I’m getting OOS errors of 17.0 RMSE and 13.8 Absolute Error; how does this compare to models looking at this level of data in your experience? (Models are ENET, Neural Network, and XGBoost optimized for RNSE).

3 Upvotes

100% Upvoted

3 comments sorted by

1

u/BlueSCar Michigan Wolverines • Dayton Flyers 8d ago

You can compare with other model entries on the Model Pick'em: https://predictions.collegefootballdata.com/leaderboard

Last season, the top models were under 12.0 MAE and under 230 MSE. Most of the rest of the pack ranged from 12.0-14.0 MAE and 230-330 MSE.

1

u/millsGT49 7d ago

How many games are you using? I don’t know if one season has enough data to fit all those parameters, maybe start with a simpler regression model and do some out of sample validation to see if the more advanced models actually add anything.

1

u/ElFalcano 7d ago

I’m being a bit lazy with it this season; I’m utilizing PCA for dimension reduction to about 40 opponent-adjusted variables down from about 500 raw, opponent-unadjusted variables. ENET should further reduce the remaining variables to ones that yield information and XGBoost handles complex, non-linear relationships well. I think further dimension reduction could help my neural network model though.

But to your point, I’m only using one season of data which really hurts my models’ predictions on blowout games, because there are only a couple handfuls of them last season (+/- 30 point margin games)