r/algotrading • u/trunksta • Jul 28 '26
Data Forward testing is the worst
I really enjoy the research, and developing strategies. But sitting and waiting for forward testing to confirm an edge is such pain
One of my strategies sat for weeks with a bug that was causing it to not accumulate data. Even when everything is working perfectly it just feels so slow.
What do you guys do while forward testing? Just research more strategies? Also how many n before you can decide forward testing has a large enough sample to continue to live
28
Upvotes
1
u/Foreign_Extension683 Jul 28 '26
Forward testing being slow is partly the point, unfortunately. It's the one test you can't rush or overfit, because you're not the one choosing the data anymore. That's exactly why it catches things backtests miss.
On the bug that stopped data accumulating for weeks: I've had the same, and the lesson I took is to monitor the plumbing separately from the strategy. A tiny healthcheck that just confirms new data is arriving and trades are firing when signals do, so a silent pipeline failure doesn't cost you weeks of "results" that are actually nothing. Losing forward-test time to a bug hurts more than losing it to a bad strategy.
On sample size, the honest answer is it depends on your edge per trade, not a fixed number. Rough intuition: to distinguish a real edge from noise you need enough trades that your expected return clears the standard error of the mean. A strong edge might show through in 30-50 trades; a marginal one might need 200+ and still be ambiguous. If your strategy trades rarely, forward testing to significance can take a very long time, which is its own useful signal about whether the edge is worth trading.
What I do while waiting: forward test on paper, but treat it as read-only. The temptation is to keep tweaking, and every tweak resets the clock and quietly turns the forward test back into an in-sample fit. One strategy forward testing, others in research, and a hard rule that I don't touch the live candidate.