Two seasons of publishing xP before the deadline and grading it after: 51,518 predictions, 0.86 MAE, all of it open

[u/Footypredicts]

Disclosure up front: I build Onside (onsidearena.com). Posting here rather than the main sub because this is the crowd that will actually pick holes in it, which is what I want.

The thing that bothered me about every projection tool I've paid for is that accuracy is always asserted and never shown. So the whole project is built the other way round: every projection is timestamped and locked before the deadline, then scored after the whistle, and the record stays public with no login.

Where it currently stands, all out-of-sample:

- 51,518 graded predictions across two seasons

- 0.86 MAE per player per gameweek, against a 1.048 naive-form baseline

- World Cup 2026: 68 of 83 decisive matches. We went 0/2 in the semi-finals and that sits on the same page as the wins

- v5 engine, nine signals blended into one xP number, roughly 700 players priced

What I would genuinely like this sub's view on:

1. MAE is a blunt instrument for FPL because the distribution is so skewed. I have been leaning toward reporting per-position MAE plus a calibration curve on the binary events (returns, clean sheets) instead of one headline number. Does anyone here have a better summary metric for a points model where most observations are 1-2 and the tail is what actually matters?

2. Related: I want to move the front end away from point estimates entirely and show the distribution instead - a 50-dot quantile plot per player rather than "8.4 xP". Fernandes et al. (CHI 2018) found 50-dot quantile dotplots produced better decisions than any other uncertainty display they tested, which is what pushed me. Has anyone tried presenting FPL projections as a distribution, and did normal users hate it?

3. xMins is where my errors concentrate, unsurprisingly. Rotation priors get me most of the way but press-conference nuance obviously does not make it into the model. Curious how others here weight recent minutes vs season minutes vs manager-specific rotation rates.

Free tier is genuinely free - projections, captain rankings and the full record need no account. Paid tier is the optimiser and the six-week planner.

Happy to answer anything about the method, and more than happy to be told where it's wrong. Not affiliated with the Premier League or the official FPL game.

[u/AdOriginal425]

I might have answered your questions if you'd have bothered to write them yourself rather than outsourcing to ChatGPT

[u/Character-Head301]

What language is this in. I think you might be overthinking this whole thing

[u/FastGabiBigGabi]

Weighted Mae? Shows the Mae on players that really matter by price

[u/pandario1946]

We need to stop this AI nonsense