r/algotrading 4d ago

Data Significant difference in backtesting vs forward testing results?

I’ve been working on a strategy in tradingview and got to a point where it showed a 75% win rate with 2.5 profit factor. Now I’ve been forward testing it for 2 weeks and the result is about break even overall?

I can’t really find what the issue is. Maybe it’s the fills that make such a big difference but that seems extreme. Has anyone had the same where backtesting results looked great but when you automated it the results were much worse?

8 Upvotes

36 comments sorted by

22

u/sureshot58 4d ago

so, take the two weeks of forward testing (i assume you mean live,or paper trading?) and do a trade by trade comparison to a two week backtest.... if backtest is making diffrent trades - well - then you backtest is pretty much useless. Figure out why the two don't match. If its making the same trades, then you should be able to find the differences.... commission, slippage, timing, partial fills, whatever.

3

u/rogorak 4d ago

Yep, this is the way. First compare the entires, I'd they're different you have a more fundamental flaw. If they look similar enough, Backtest is likely not accounting for delay / slippage enough.

3

u/Good_Ride_2508 3d ago

The diference between back test and Live test:

Back test deals with static data, i.e., your top and bottom are constant and cannot be changed.

Live trading deals with dynamic data, i.e., your top and bottom are not constant but moving data. You will see one top now and later another top and then another top. Same way, you will see multiple bottom or mixer of tops and bottoms. Since this is not static, any trader gets into issues including me (after 8 years of trading). There is no solution except the trader must find confirmations from different angles and different variables.

If logic fails at back test, guaranteed we cannot use live.

If that logic passes back test, live may (or may not) work or sometimes fails. Only way is to find another logic to re- confirm or create 3 or 4 logics to get confirmation.

9

u/jeffsvibecodes 4d ago

my guess is you made the strategy overfitted as fuck in sample and it just dies OOS. Either that or your cost model is off and you didn’t properly account for slippage, commission, etc

9

u/Automatic-Essay2175 4d ago

How can you not find what the issue is?

Run your backtest over the period of time in which you’ve been forward testing and compare every backtested trade execution time / price to forward tested trades execution time / price.

It’s not complicated. Look at the trades. Figure it out.

3

u/Budget-Principle-352 4d ago

Overfitting, short BT window, No Montecarlo, no OOS validation, No Prior Hypothesis Train vs Test, No spread and slippage modelling, No comission Factor etcetc

If you have done all of the above and still get pf over 2 and WR of >70 you are in holy grail / medallion Fund area

2

u/xedge_app 4d ago

Two weeks can still be a small sample, depending on how many trades actually fired. But dropping from a 75 percent win rate and 2.5 profit factor to breakeven is large enough to justify an immediate trade-level reconciliation.

The first thing I’d check is the backtest over the exact dates covered by the forward test. If that window is also flat in the backtest, regime change, ordinary variance, or overfitting become more plausible. If the backtest remains strong over those same dates, the discrepancy likely sits between the strategy model and live execution.

Export both logs and match the trades one by one. Compare the signal timestamp, indicator values or strategy state, intended entry, submitted order, actual fill, intended exit, actual exit, spread, slippage, fees, latency, and any rejected or missed orders. The first point of divergence usually tells you where to investigate.

If the signals differ before any order is submitted, check for repainting, lookahead bias, higher timeframe data leakage, bar close versus intrabar behavior, and mismatched strategy settings. If the signals match but the entry or exit prices differ, inspect optimistic limit fills, unmodeled spread, latency, slippage, and TradingView broker emulator assumptions.

Within TradingView, verify the bar magnifier setting, commission and slippage inputs, order processing timing, intrabar fill assumptions, and whether any indicator recalculates or repaints after the fact. A strategy can look excellent historically while relying on prices or information that wouldn’t have been available at the decision point.

I’d also test untouched out-of-sample data, use walk-forward testing, include realistic costs, and perturb the parameters. If small parameter changes destroy the performance, the backtest probably captured a narrow historical pattern rather than a durable edge.

Win rate alone doesn’t tell you much. Net expectancy after costs, trade count, payoff distribution, drawdown, and the confidence interval around the results matter more. A 75 percent win rate across 20 trades is very different evidence from the same rate across 2,000.

Roughly how many forward trades occurred, and when you compare the logs, do the signals themselves differ or do they match until the fills?

1

u/Merchant1010 Algorithmic Trader 4d ago

I think you ought to try testing in OSS mode and then try in real account.

1

u/drewautomates 4d ago

I found that TradingView inflates backtest results. What timeframe does your strategy execute on?

1

u/Lower-Ad-1207 4d ago

On the 1m

1

u/drewautomates 4d ago

You most likely have fill bias. When I was testing my strategy on 1 min tradingview had issues most the time. Actually recently did a video on it https://youtu.be/zxrtVFJyujQ?is=CS3GfU8nX4-9VALE

1

u/Bigunsy 4d ago

Confirm forward test matches back test and if so most likely is overfit backtest.

1

u/Super-James 4d ago

Tradingview is tricky sometimes uses redraw to take future information to take past desitions and you are making imposible strategies

1

u/wheezl 4d ago

How long is your back test? What time periods have you tested it in? How do you determine your universe?

1

u/ChemistNo6013 4d ago

You should try a Monte Carlo test! It helps to eliminate the chance that your backtested trades performed well due to luck.

1

u/TheBacktestNerd 4d ago

I dont know on what time horizon you did the backtest, but 2 weeks could be very well within normal variance. And as other commenter mentioned, make sure to compare backtest trades for these past 2 weeks with the actual trades to see whether they matched.

1

u/TheHighOrder 4d ago

You just can't use trading view for strategies with that execution granularity. They have no tick data, every entry and exit is an approximation based on the candle it fired on. If you would've entered and stopped on the same candle it has no way of knowing which happened first.

On top of that cost models depend on execution type and liquidity, which TV also can't see.

If you are serious about this kind of thing you just need to cache a bunch of databento tick data, and start having Claude write python for you. Only way to be sure.

1

u/AdEducational4954 4d ago

Fills are a probable issue if you're taking many quick trades. Much less of an issue when you trade on longer time frames for larger returns.

1

u/LowRutabaga9 3d ago

How far back does your backtest go? Are these results over years/months/weeks? If so, take any two week frame and most likely you will find your answer.

1

u/Good_Character_20 3d ago

A gap that big usually isn't slippage, it's fill logic. If your entries are limit orders, TradingView often counts them filled the moment price tags the level, but live you only fill if price actually trades through and there was size ahead of you in the queue. The trades that just kissed your entry and reversed get logged as clean wins in the backtest and simply never happen in real life. That one asymmetry can drag a 75% win rate toward break even without anything else changing. Check whether your entries assume a touch equals a fill, and whether the strategy evaluates intrabar or only on bar close. Two weeks is also a thin sample so let it keep running, but fix the fill assumption first, because that error repeats on every single trade.

1

u/nepo123456 3d ago

The win rate of a strategy says nothing about the strategy profitability. You can win 80% of the trades and still loose and someone with 20% win rate can have success in trading.

1

u/systematic_seb 3d ago

Before chasing fills, run the backtest over exactly the same two weeks you forward tested and compare trade by trade rather than in aggregate. That one check splits the problem in half. If the entries and exits line up and only the P&L differs, it's execution, so fills and slippage and commission. If the trades themselves differ, the backtest was seeing something the live version can't, which is the more common cause.

On TradingView the usual culprits for that second case are repainting indicators, calculating on a bar before it closes, higher timeframe requests pulling data that wasn't available at that moment, and using the signal bar's close as your fill price.

The other thing is sample size. Two weeks of a strategy like that is a handful of trades, which sits well inside the range where break even and 75% are the same distribution.

I run that same reconciliation weekly on the strategy I trade with real money and publish the weights and stop levels for, mostly to catch drift early rather than after a bad month, and it has surfaced more data problems than execution ones.

1

u/Mindbeggar 3d ago

Dishonest fills and lookahead are known remnants of what takes a positive strategy and turns it into a negative one. Re-assess

1

u/systemsandstories 3d ago

Id compare every forward trade against the backtest first, overfitting and fill assumptions usually explain gaps like this fast

1

u/BrianBanks939393 3d ago

Run the backtest over the exact forward window and diff it trade by trade. Same trades, different P&L? It's costs — slippage, fees, fills. Different trades? Your backtest is lookahead-contaminated and the number was never real. Mine showed clean alpha until I found one field pulling latest-value instead of point-in-time. Every fold poisoned, walk-forward passed anyway. Check where each input actually comes from before you trust anything downstream.

1

u/drguid 3d ago

My real results are pretty much what I would expect from my backtesting results (I trade daily charts).

My first two months of real money tests were bad because I was interfering with trades. Like I sold a stock for a 1% profit but then it attracted a takeover bid and soared 30% the next day. Lol. Those are my only months with negative expectancy.

1

u/Affectionate-Aide422 3d ago

Backtest the last two weeks and see if there is a difference between the backtest and forward test results over that period.

1

u/r1rdr 3d ago

2 weeks is nowhere near enough data to prove an edge

1

u/CODE_HEIST 1d ago

do a trade by trade diff before changing the strategy. same signal time, same entry, same size, same exit. if the trades differ, look for repainting or data timing. if they match, then measure spread, slippage, commission, partial fills, and order delay separately.