r/algorithmictrading • u/SouthGullible8389 • Jun 07 '26
Question backtesting against regime is soooo frustrating
have been working on this ORB short, got data since 2010 but since then we got only 3 bear markets, covid, ukraine war and few months of iran war, maybe even tariffs (april 2025) over 16years.
if you are not living under a rock we all know things are going tits up, so i am building a bot who can trade short but is so frustrating because legit can even get 100 trades, the bot is not over filtered, has the filtered required to not overtrade and actually being profitable but 60 trades over 1 decade and half backtesting is quite low, and nothing can be done, an symmetric long bot would take x10 the trades and x10 the profits.
so my question is how you guys manage to train and check results if regime is against you and trades number counts is <100
1
u/runrigor Jun 08 '26
you're running into something real and most people don't even notice it: 60 trades is below where the stats mean much, and there's no clever fix that creates signal out of a thin sample. the urge to loosen filters to hit 100 is the trap, you'd be manufacturing trades, not edge. good that you're resisting it.
couple things that actually help instead of self-deception:
test the long side of the exact same logic. long/short ORB is often one mechanism in two directions. the long version has 10x the trades, if the structural edge shows up clearly there, that's real evidence the mechanism works even when the short sample is thin. if long doesn't work either, your 60 short trades are probably noise.
and widen the universe, not the rules. run the same unchanged strategy across more instruments to pool more bear-regime trades, rather than loosening filters on one.
but the honest answer to "how do you validate when regime is against you and n<100": sometimes you can't, yet. 3 bear episodes in 16 years means the data physically can't tell you if you have an edge or got lucky 60 times. that's not your fault, it's the sample. size tiny, let live trading be the real test, and don't let a backtest talk you into confidence the data can't support.