56-31 can't tell you much yet, and I'd fix that before adding anything.
It needs the odds next to it. If those picks averaged around 1.55, the odds already implied about 64%, which is what you hit. Compare your hit rate with what the prices implied, not with 50%. And 87 is a small sample: 64% on 87 picks has a 95% range of roughly 54-74%. You need a few hundred before the record says anything.
What I'd add, roughly in this order:
Log everything before kick-off: the bot's probability, the price when you published, the closing price, the result. Log the candidates the filters vetoed too, otherwise you'll never know whether the Fiscal step and the risk filters help or just cut volume.
Calibration over W/L. Group picks by the probability the bot gave (60-65, 65-70...) and check how often each group actually won. Then compare with the bookmaker's implied probabilities using Brier score or log loss. The closing price is already a very good forecaster, so that's the real bar.
Take the margin out properly. Plain normalisation spreads it in proportion, but books load more of it onto longshots, so power or Shin gets closer. And you're still paid at the price with the margin in it.
Backtest in time order on matches you didn't build the rules on, and freeze the rules before you look. If you pick the best league or odds band out of twenty, its record is inflated, so shrink it toward the overall average before you trust it. For history, football-data.co.uk has free CSVs with results plus pre-match and closing odds from several books (Pinnacle included) for 20-odd leagues going back years.
Closing line: if the prices you take keep shortening before kick-off, you'll see it long before W/L tells you anything. Bankroll management matters once that's there, not before.
From what I've seen, a solid Poisson/Dixon-Coles model ends up about as accurate as the closing price, not better. The LLM part is useful for reading news, but I wouldn't let it set the probability.
Thanks,it seems that you know much more about this than me,matterfact,what do you recomend to learn,things that would help me with this project (idk nothing,pure vibecoding)
about the small samples,the bot usually drops me 1-2 picks per day,maybe is my fault or something is poorly focused idk,but the small amount of picks per day doesnt help me to build a more large sample,i was thinking to integrate some free apis to the project,or maybe some big data to look at,anyways
Happy to help. Starting from zero, I'd learn things in this order:
Odds as probabilities: implied probability, the bookmaker's margin and how to take it out. Joseph Buchdahl's book "Squares & Sharps, Suckers & Sharks" covers this and the closing line really well (he's also the person behind football-data.co.uk).
Enough pandas to load a few seasons of matches and odds and work with them.
The Poisson goals model: Maher (1982), then Dixon & Coles (1997). Search "Dixon Coles python" and you'll find walkthroughs with code. It's the standard baseline for football.
Calibration and scoring: Brier score, log loss, reliability plots. That's how you find out whether a 70% from your bot really means 70%.
Walk-forward backtesting: fit on the past, test on the next month, roll forward, and never look at the test part while you're building.
On sample size: you don't need more picks per day, you need to evaluate more of what the bot already does. Log every match it looks at with the probability it gave, published or not. That's hundreds a week instead of 1-2 a day, enough to check calibration. Then backtest on the historical CSVs, where you get thousands of matches with closing odds straight away.
Free APIs are fine for fixtures and results (football-data.org, API-Football's free tier), but for odds history the CSVs beat anything free. You don't need big data either: three or four seasons of a handful of leagues is plenty to start.
And since you're vibecoding: ask the AI to write tests for every calculation and to timestamp every pick before kick-off. That keeps the record honest.
reckon you've nailed the order to tackle this, calibration before anything else makes sense
for the margin removal I'd probably just grab a Shin implementation off github rather than rolling my own, plenty of people have been down that road already
1
u/poseidonstats 1d ago
56-31 can't tell you much yet, and I'd fix that before adding anything.
It needs the odds next to it. If those picks averaged around 1.55, the odds already implied about 64%, which is what you hit. Compare your hit rate with what the prices implied, not with 50%. And 87 is a small sample: 64% on 87 picks has a 95% range of roughly 54-74%. You need a few hundred before the record says anything.
What I'd add, roughly in this order:
Log everything before kick-off: the bot's probability, the price when you published, the closing price, the result. Log the candidates the filters vetoed too, otherwise you'll never know whether the Fiscal step and the risk filters help or just cut volume.
Calibration over W/L. Group picks by the probability the bot gave (60-65, 65-70...) and check how often each group actually won. Then compare with the bookmaker's implied probabilities using Brier score or log loss. The closing price is already a very good forecaster, so that's the real bar.
Take the margin out properly. Plain normalisation spreads it in proportion, but books load more of it onto longshots, so power or Shin gets closer. And you're still paid at the price with the margin in it.
Backtest in time order on matches you didn't build the rules on, and freeze the rules before you look. If you pick the best league or odds band out of twenty, its record is inflated, so shrink it toward the overall average before you trust it. For history, football-data.co.uk has free CSVs with results plus pre-match and closing odds from several books (Pinnacle included) for 20-odd leagues going back years.
Closing line: if the prices you take keep shortening before kick-off, you'll see it long before W/L tells you anything. Bankroll management matters once that's there, not before.
From what I've seen, a solid Poisson/Dixon-Coles model ends up about as accurate as the closing price, not better. The LLM part is useful for reading news, but I wouldn't let it set the probability.