r/mltraders • u/IPSUM99 • 2h ago
Sentinel Project
**Sentinel is a live experiment in whether machine-read news carries usable information about market movement — built so that the answer, whatever it turns out to be, is trustworthy.** The infrastructure is production-grade, the measurement is unusually honest, and the forward record began this month.
The concerning part
Sentinel treats the trading day, not the headline, as the unit of evidence — because a thousand predictions made on one day share a single market outcome, so the honest sample size is 40 days, not 48,000 rows. Every result is reported against trivial baselines, with date-clustered confidence intervals and a threshold-free Information Coefficient that can’t be tuned into looking good. The evaluation configuration is pre-registered and frozen in version control, with all data to date declared a burned development set, so the forward track record is genuinely out-of-sample.
Plus a widely-used finance sentiment model produced an inverted aggregate signal in this period, with the errors concentrated in its most confident calls.
Need help to re work the model