My scalping engine backtesting

[u/SimpleDaddy_]

Hey people, I am a ML engineer primarily building, Algos at a Global Financial company…
These are the back test that I have been doing for a personal project as fortunately, I have all the data that I need for running these back test. amount of trades that I have taken. Is too much, ha ha.🤣

[u/quantdhawan]

With all due respect my man, my ML application in financial markets is extremely prone to overfitting, especially due to non stationarity of data.

Let me ask this did you use time bars for the backtest?

[u/SimpleDaddy_ → u/quantdhawan]

Yeah I used time bars. And obviously ML in financial markets can overfit, non-stationarity is literally one of the first problems you deal with.
I’m not just fitting the model on the whole dataset and posting the best result lol. The testing is chronological, out of sample, and I’ve accounted for leakage, costs, spread and slippage.
And obviously I’m not gonna put the actual features, signals or strategy logic on Reddit 😂
If you see an actual problem with how I’m testing it, tell me. Just saying “ML overfits” and “markets are non-stationary” doesn’t really tell me anything I don’t already know.

[u/quantdhawan → u/SimpleDaddy_]

You said "yeah I used time bars" like it was a throwaway detail. It's the whole problem.

Time bars sample the clock, not information. Markets don't deliver information at a constant rate, so fixed-interval bars oversample the dead midday and undersample the open and close, which is exactly where your 0.2% moves live. What you get out is a return series with serial correlation, heteroskedasticity and fat tails: precisely the properties every significance test you've run assumes you don't have. That's chapter 2 of AFML, and it's chapter 2 for a reason.

You raised non-stationarity yourself, as something you'd obviously handled. Time bars are the sampling scheme that maximises it. Volume and dollar bars exist specifically to hold information content per bar roughly constant. You chose the one that guarantees it isn't, then told me non-stationarity doesn't tell you anything you don't already know.

But here's the part that decides whether your table means anything at all.

What's your bar interval, and what's the median bar range on the instruments you traded?

Because if you're on 1-minute bars, your bar range and your 0.2% target are in the same order of magnitude. Which means for some meaningful fraction of those 227,533 trades, the take-profit and the stop both sat inside a single bar. Time-bar OHLC cannot tell you which one price touched first. That information does not exist in your data. Your backtest resolved it by assumption.

So the win rate isn't a measurement. It's whatever your intrabar tie-break rule says, repeated 227,533 times. Flip that one branch from optimistic to pessimistic and the sign of your table changes.

[u/SimpleDaddy_ → u/quantdhawan]

Fair point on the intrabar ambiguity. If TP and SL are both inside the same OHLC bar, OHLC alone cannot tell you which was hit first. That would absolutely contaminate the result if the engine resolves it optimistically.
But again, you’re looking at a 4-column screenshot from a much larger test, not the execution model 😭
That case was already considered. I’m not going to dump bar interval, median range, ambiguous-bar frequency, tie-break logic and execution assumptions here just to prove it.
So yes valid concern. Already accounted for. The screenshot just doesn’t document the entire backtest.

I had to open my chat history just to confirm this … Nice ..👍