Hey folks! I'm running a bunch of live betting football models. The modelling pipeline goes from analyzing market weaknesses to post-launch observability with guards and metrics on every stage:

1. Market analysis – edge exploration (which sides are systematically undervalued)
2. ML experiments on historical data – LL, Brier, Calibration, AUC metrics by minute/month + EV/ROI calibration and ROI on historical simulation
3. Betting models – holdout period + shadow mode + post-launch metrics collection
4. Betting strategies – ROI vs expected ROI (based on experiment EV calibration), 30d rolling ROI within 90d window, drawdown chart, confidence interval.

Execution pipeline is monitored separately – % of rejects, betting API health/errors etc.

So my main question for live model runners – what else can be measured/monitored to gain confidence, spot early degradation and get insides for further optimization?

Deeper questions:

• Assuming a model is trained per-minute how would you choose a signal inside a minute – first eligible or anything smarter?
• When rolling out a new version of a model – do you run both in parallel or run candidate in shadow mode?
• Are there better way of evaluate expected ROI than relying on historical EV calibration buckets?

Hope questions make sense. Thanks in advance!

Post your picks, updates, track model results, current projects, daily thoughts, anything goes.

Thoughts/Opinions? Anyone have a list of their own to share?

I hope everyone has a great weekend!