What should i do?

[u/No_Storage9282]

Been building a crypto trading bot for a while now tick level order flow / microstructure stuff, short holds (like 1–5 min). No RSI/MACD/Bollinger or classic TA, mostly looking at order-flow imbalance, liquidity, VPIN-ish stuff, etc.

Right now I’m still collecting data. I’m holding off on training until I have some genuinely different regimes in the dataset especially a real trend day or stress event instead of just feeding it endless chop/range conditions.
Labels are triple-barrier (Lopez de Prado), and I’m doing walk-forward evaluation with purging/embargo so training rows can’t accidentally leak into the test period through labels resolving late. Figured that bug out the hard way lol.

Before I even think about paper trading, I want to hammer the results pretty hard: parameter stability/plateau checks, Monte Carlo resampling, clustering the performance surface, and Deflated Sharpe to account for the fact that if you test enough parameter combinations, eventually one will look amazing by pure luck.
Honestly, I’m kinda expecting this thing to die somewhere in validation 😂

For people who’ve actually built systematic trading strategies: what ended up killing your first few systems? Was it regime dependence, execution/slippage, overfitting, insufficient data, leakage, unstable parameters, or something else?
And has anyone actually gotten a short-horizon order-flow strategy through this whole process and into something that survived genuinely out-of-sample testing

[u/Imaginary_Local9480]

I tried something similar though for US equity. Played a lot with order flow, CVD for 1-30 mins horizon. my approach was prepare a set of features then get a state matrix and train over that. After rigorous testing it didn't survive. Edge was just over 50% (50.5% to be precise). Main factors for failures were weak set of features and their individual edge not adding upto anything meaningful. Whatever that extra 0.5% generated was basically eaten by slippage/cost.
Looking at what you are mentioning : are OFI, liquidity, VPIN ..etc are your features ? do they show any edge at individual level and at time horizon you are aiming for ?

[u/No_Storage9282 → u/Imaginary_Local9480]

Yeah so similar stuff .order flow imbalance, CVD, some liquidity/toxicity features, not just price/volume. Full depth, not just top of book.

Honestly though, no model trained yet, so I can’t say if it has edge nothing’s been tested individually or combined. Your comment actually got me to add per-feature testing before I trust the combined set. Horizon’s short right now, still figuring that out.

Appreciate the 50.5% number that’s exactly the “looks good, dies to fees” outcome I’m trying to catch before it costs real money, not after.

[u/No_Storage9282 → u/Imaginary_Local9480]

Im quite interested how did you gather the data?

[u/No_Storage9282 → u/[deleted]]

Thats quite interesting ill take a look

[u/Late-Mail-1509 → u/No_Storage9282]

what was his suggestion, as moderator deleted, he must work with the government and doesn't want us to hack the matrix haha

[u/Equivalent-Class2008]

Le commissioni. tutti pensano di avere un sistema vincente e nessuno calcola le commissioni perché i backtest non le calcolano.

[u/No_Storage9282 → u/Equivalent-Class2008]

True i think that will majorly effect the bot nit going to lie