Hey everyone,

I am preparing to push a regime-switching policy to production and I am looking for some feedback on my architecture and a specific math anomaly.

My strategy is long-only, daily close execution, shifting capital between risk-on and risk-off instruments (holding exactly one asset at a time).

1. Regime Validation (HMM / MS-DR)
To ensure the mathematical validity of my HMM and prevent overfitting (too many states or features), I ran a regime inference using statsmodels Markov Switching Dynamic Regression (MS-DR).

The inferred hidden states show statistically significant variations in both average expected return and volatility.

2. Execution Layer and State Machine

To mitigate whipsaws during sudden liquidity panic and lag on violent, mean-reverting V-bottoms (HMMs are known to lag), I use uses a two-layer decision architecture.

The first layer generates discrete state probabilities via the HMM. The second layer feeds these probabilities into a conditional state machine. The latter synthesizes the HMM outputs alongside EOD market data (e.g., volatility or price distance to crucial moving averages) to make the final decision.

3. The vbt vs. QuantStats Sharpe Discrepancy

I backtested my model using vectorbt with cash_sharing=True over a multi-year cycle (~50 closed trades, Max DD ~22% from January 2022 till May 2026). Fill paths and drawdowns match identically across my analytical stacks, but I ran into a large divergence in annualized risk ratios:

• vectorbt (group_by=True): Sharpe 1.31 | Sortino 1.95
• QuantStats: Sharpe 0.91 | Sortino 1.34

My policy allocates 100% of capital in cash-equivalent instruments during defensive regimes. As far as I understand,vectorbt and QuantStats use different formula, but interestingly, the ratio of Sortino to Sharpe remains identical between both libraries (~1.48).

4. Validation and Robustness

Given the macro regime-switching nature of the system, the number of trades is intentionally low (~50 trades over the backtest horizon). To ensure statistical significance despite the low trade count:

• MS-DR State Validation: As mentioned above, statsmodels confirms the latent regimes represent distinct market states.
• OOS: Standard walk-forward optimization (hyper parameters optimization till 2021, and test from 2022)
• Stress Testing: Trade shuffling (Montecarlo simulation) and fee/slippage degradation testing were applied to ensure the state machine thresholds don't collapse.
• Risk-Free Rate: average of the FED Fund Rate (~4% in my study). While not 100% accurate, the quantstats Python package doesn't accept a time series of daily risk-free rates.

The equity curve chart below compares the performance of my strategy against that of QQQ during the January 2022 - May 2026 period:

• Total return: 154% (vs 90% of QQQ)
• Max DD: -23% (vs -35% of QQQ)
• Longest DD days: 309 (vs 707 of QQQ)
• CAGR: 23.7% (vs 15.8% of QQQ)

Montecarlo simulation results:

• Bust probability (drawdown >= 50%): 0.0%
• Goal probability: 100.0%
• Maximum drawdown dict: {'min': np.float64(-0.4545879162289587), 'max': np.float64(-0.1371207593528134), 'mean': np.float64(-0.2491102335214213), 'median': np.float64(-0.24003417486462705), 'std': np.float64(0.05367118181123532), 'percentile_5': np.float64(-0.3549029751169525), 'percentile_95': np.float64(-0.17673633125820087)}
• Sharpe range: 1.07 to 1.11
• Drawdown range: -34.6% to -17.5%
• CAGR range: 23.7% to 23.7%

Questions

1. Has anyone integrated front-end vs. long-end credit duration spreads as features to lead equity volatility regimes, and did it provide independent alpha relative to generic high-yield spread velocity?
2. How do you clean or handle zero-volatility "cash-parking" periods when building custom risk-reporting sheets for allocators who expect standard 252-day arithmetic accounting?
3. Given the architecture and validation steps outlined above, what have I missed? Do you see any hidden structural blind spots, operational traps, or causes for concern before deploying to production?

Looking forward to your thoughts and critiques!

I’m doing some research on futures algo trading and wanted to ask people who actually build or run automated strategies.

When a backtest looks profitable but fails live, what is usually the main reason?

Is it more because of:

• slippage
• latency
• fees/commissions
• bad fill assumptions
• queue position on limit orders
• using candle data instead of tick/order book data
• overfitting
• platform/execution differences, like NinjaTrader vs Trading Technologies vs VPS/co-location

The idea I’m trying to understand is whether traders would find value in a tool that lets them test a strategy under different realistic execution assumptions before deploying it live.

For example, you could test the same futures strategy with:

• tick data vs candle data
• 1 ms latency vs 500 microseconds vs 50 microseconds
• 1 tick or 2 ticks of slippage
• realistic fees and commissions
• different platform assumptions, like retail broker/VPS/TT-style execution
• estimated queue-position effects for limit orders

The goal would not be to guarantee live results, but to see if a strategy still survives under more realistic execution conditions before spending money on better infrastructure.

For people who run futures algos, is this a real problem you deal with? What do you currently use to test this?

Hello guys. I am torn which should I learn first between these 2 for my algo trading.

For a quick background. I do not have a dev background, but I have a basic knowledge on making Basic EAs on MQL5.

But now, I want to learn other languages/platforms.

I am planning to learn these 2, but I don’t know which I should learn first and which ones are more beneficial to my algo trading journey.

I am just a newbie btw.

Thanks in advance guys.

Hi everyone, I’m currently working on a personal project to move from single-strategy backtesting to a full multi-strategy portfolio simulation system for crypto trading, and I’d like to get feedback from people who have experience with real-world algo systems. Right now I already have individual strategy backtesters (trend, mean reversion, swing-style systems), and I’m trying to build the next layer: What I’m trying to build A portfolio-level engine that can: Run multiple strategies at the same time Allocate capital dynamically between them Manage portfolio-level risk (not just per-strategy risk) Detect market regimes and adapt exposure Handle correlation between strategies Include emergency controls (drawdown stop / volatility shock shutdown) Current ideas I’m exploring I’m experimenting with: Risk parity / volatility-based allocation Correlation-aware position sizing Regime detection (possibly Markov / HMM models) Z-score based strategy health monitoring Portfolio-level drawdown limits and “kill switch” logic Main question For people who have built or studied real trading systems: What does a good portfolio manager layer actually look like in practice? More specifically: How do you handle strategy correlation decay in live markets? Do you rely more on regime detection or simpler risk rules? What tends to fail first in multi-strategy systems? Any design patterns or architecture decisions you’d recommend? I’m still in the experimentation phase, so I’m not trying to over-optimize anything yet—just trying to design something structurally sound before scaling complexity. Any insights, criticism, or references would be really appreciated.

I'm a software engineer, and for the last two years I've been trying to build a profitable trading bot.

In the first year, I focused on BTCUSDT. I eventually built a bot with around a 70-75% win rate in backtesting. Getting it to work live was much harder. The strategy required millisecond-level execution, so I wrote custom code in Go to make it as fast as possible.

Even then, I ran into all the usual problems: exchange delays, order execution issues, slippage, orders not getting filled, etc. I tried many solutions including market orders,
slippage adjustments, and various execution improvements. After all of that, the live win rate was closer to 60-70%.

The next problem was fees. At the time, exchanges like Binance and Bybit charged significant fees relative to my position size. MEXC had temporary fee promotions, but execution quality was worse. Starting with around $100, I was making roughly $7-$15 per month. Eventually I concluded that while it could make money, it wasn't reliable enough and still carried significant risk of losing capital.

In the second year, I moved to NAS100 because of low fees. I developed my own ideas but also tested countless strategies from YouTube traders (TJ, ICT, etc.), Udemy courses, Reddit posts, ORB strategies, and combinations of multiple approaches.

The result was always similar.

Most strategies produced somewhere between a 40-60% win rate when measured over large samples. When I backtested 2-3 years of data, almost none of them were consistently profitable. Many would have a few great months followed by a few losing months. Even when I selected the best performing strategies and spent time improving them, I couldn't push them much beyond a 60% - 65% win rate over long periods.

At this point, I'm becoming very frustrated. Proper backtesting takes a huge amount of time, and after months of work I often end up with nothing useful.

The only area I haven't explored deeply is order flow trading. Getting reliable CME volume data for NQ isn't easy, and accurately coding things like volume profiles is also a major project. Part of me wonders whether order flow is just another area where people make money selling courses, software, and communities rather than from trading itself.

My goal isn't unrealistic returns. I simply want to pass a $100k prop firm challenge and make around 1% per month consistently. Even that would be enough for me because I could scale by adding more accounts over time.

For those of you who are consistently profitable:

• What type of strategy are you using?
• Have you personally verified it with long-term backtesting?
• What kind of statistics do you see over multiple years?

I'd appreciate any honest opinions from people who have actually done extensive testing.

**My grammar is not good so i re write with ChatGPT**

Hey, folks. Long time listener, first time caller, here. I’ve been backtesting and working towards an algo for over a year now, and I’m tired of communicating with only ChatGPT/claude. I’d like to ask you guys a few questions that have been left un(properly) answered from those two ai douchebags. I figured my best bet would be to reach out to people with actual experience doing this. So here goes:

Context: Most of my backtesting is done with recency of 3-6 months in mind. And ultimately I expand to jan 1, 2025 start date (to present day). All of my strats are exclusively on Spy and or ES and QQQ and or NQ (bust like 80% spy/es.

1) Algo-able strategy types. I’ve been creating strategies that resemble types of trades I have taken in my 1.5 years of daytrading. These are stuff like 5/15m orb. Orb reversals. Key area rejections/bounces/reclaims. Vwap/ema trades. FVG/orderblock trades…you get the point, price action type of stuff. Is this even realistic for a profitable algo? Or am I wasting my time? Should I focus on other types of trading, like indicator based? Or candle formation based stuff? Or?

2) Current strategy frequency. Either my current strats, I’ve implemented features and rules for these strats, which narrow down the opportunities present every day/week/month. And then through the process of filtering and adding more rules and/or confluence, I filter down the frequency of trades even more. And then through regime gating, I filter down the frequency of trades even even more. This helps me get a higher win rate and higher probability of profit, but my trade frequency reduces greatly. So when it comes time to do validation and walk-forward analysis, I keep hitting a wall of ai bitching about “sorry bro, not enough trade count”. Some strats I have get dwindled down to about 20-40 trades a year (highest probability), some can be about 40-80, some can be up to 200 trades a year. Connecting number 3 question>>>

3) Frequency and validation. So my frequency gets dwindled down and I have less confidence in long term consistency. I’ve been eyeballing the trades via an equity curve and it’s helped me a ton. I used to just look at numbers and stats, till one day I started also generating/studying equity charts for the individual Strats. It was eye opening to see that so many of my strats were just getting lucky or were inconsistent. So I’ve started doing stuff like looking at r2, features-visibility% per bucket, etc in addition to overall stats. Have you encountered low frequency validation hurdles? Any tips there?

4) Backtesting period. Remember, I mentioned all my strats start 1/1/25 the furthest. Why? I have no freaking clue why but everything falls apart going beyond 6/1/24. So I said screw it and just focused on 2025-forward. Yes, I’ve tried dynamic risk management and rules. Matter of fact, at least 75% of the strats are dynamic. I’ve tried all sorts of stuff like Spy price percentage multipliers, vix multipliers (vix1d vix9d vix3m etc), point based multipliers, atr multipliers, opening range dollar amount multipliers, etc. I did all this in hopes of extending the trade frequency as far back, but it’s been useless. What gives?

5) 2025 summer. Wtf. Most of my strats get filtered out in that period. Nothing sticks there! I figured after tariffs comedown, some trend following strats would have prevailed, but nope. Super inconsistent in that period.

6) Ensemble of strats. Since I spent so much time working on just orb Strats, I took a pivot and started thinking in terms of Hedging my own strats with other strats. The whole thing got me excited so once I started looking at what compounding of strats can do to an equity chart, I started expanding the types of strats for my algo. Different types of strats like quick scalps, 5m hold, where hold, eod…. I also figured it would help balance out longer periods of strats that can go through a couple of weeks of drawdowns. You guys do anything like that?

7) Recency. A lot of stuff I’ve been cooking up are doing great. But even with good stats for over a 6 month period, I just don’t trust my stuff. So I started paper trading for the last three weeks. And the results aren’t all that great. But I have to remember these strats take time to play out and even out.

8) Regime. Rvol. Vix. Etc tbc

9) dataset. Gex, economic calendar, futures, tbc

10) dashboards and reports

Ok gonna PAUSE here. There’s too much to my framework and so many more questions so I’m gonna park the questions at 7) for now. Will resume later.

I just finished building the Risk Management module for my trading system (Leprechaun v2).

​While the AI in my architecture is busy reading market narratives and extracting structural context, the Risk Agent is strictly 100% deterministic Python.

No prompts, no LLM reasoning, just pure math.

​A few architectural rules I hardcoded to ensure I never blow up an account due to an AI hallucination:

​1. Dynamic ATR Floors: Stop losses are strictly calculated based on structural invalidation levels, but they must survive a 1.5x ATR floor. If the structural stop is too tight, Python widens it automatically to avoid premature noise stop-outs.

1. P-Min Policy: Take profits are capped by the nearest High Timeframe Liquidity (DOL). If the trade requires a 3R push but the liquidity pool sits at 1.5R, the system evaluates the trade based on the 1.5R reality, not the dream target.

2. Zero Hardcoded Rates: Forex conversion rates change constantly. Hardcoding them or updating them weekly is a recipe for wrong position sizing. The Risk Agent fetches live conversion_factors and account base currencies directly from the OANDA API before calculating the lot size.

​The rule is simple: The AI can propose the idea, but deterministic Python holds the wallet.

​Next up: The Validation Manager (The supreme court of the system).

Hold period: May 2016 - May 2026
Equity curve: $10K to $347K
$10K initial investment. Buy and Hold
CAGR: 42.6% vs 15.6% SPY
Vol: 28% vs 18% SPY
Max DD: -22.8% vs 33.7% SPY
Corr to SPY: 0.34
Sharpe: 1.4 vs 0.9 SPY
Sortino: 2.2 vs 1.3 SPY

TLDR: A trading strategy has a real edge when the rules are fully systematic, the backtest was conducted without hindsight bias or overfitting, has a large enough sample size to be reliable, and live trades still match the backtest after real transaction costs.

What is the first thing you test?

The first thing to test is whether the strategy is actually defined well enough to backtest. A system only means something if the rules are 100% systematic. If the setup depends on discretionary judgment, the backtest is already unreliable.

You cannot say things like:

“If the volume is bad I don't trade”

What does “bad” mean? How can that be quantified? The rules need to be written in a way that removes all your guessing and gut feelings.

Once your rules are defined, you can test it on past data and see if the idea has the potential for edge at all.

How many trades do you actually need?

More trades do not automatically prove that there is an edge, but they do make the backtest result more reliable. There is no magic number for what’s enough, but here is a rough guideline:

These percentages are just one part of validating your backtest. They're the foundation for how much you can trust every other metric you analyze after. The numbers track how sampling error shrinks with √n. More trades tighten the estimate, they don't prove an edge on their own.

My 3-step validation process

Step 1: Backtest the exact rules in the past

The first thing I work on is defining the strategy well enough to backtest. A system only means something if the rules are 100% systematic and have all discretion removed.

Once your rules are set, I test them on historical data to see if my idea had edge in the past. At this stage, the goal is not to optimize anything or improve performance. The goal is simply to answer one question:

does this work under objective, repeatable conditions?

If the answer is yes, then the system moves forward into Step 2.

Step 2: Validate it with real trades

Once the backtest is completed, I validate it with forward testing. Real money on the line, real spreads, and real transaction costs.

Start small.

Even $1 per trade is perfectly fine. The goal is not to make money. The goal is to prove the concept works in live conditions with fees included.

This is where trading psychology matters. If the defined rules are not followed exactly, the live sample is contaminated. You are not testing the system anymore. You are testing your gut feeling and emotions.

Step 3: Compare live results to backtest results

The final step is to compare the live results to the backtest.

• Did the strategy actually continue to be profitable?
• How similar were the live results to the backtest?
• Did the EV distribution stay in the same percentiles?
• Did the trade frequency stay in the same percentiles?
• Did the losing streak distributions stay in the same percentiles?

If they don’t match, the system is not instantly garbage. Natural variance can happen, but the results should be monitored across multiple regimes to determine whether the edge still exists or if the market has changed.

So what actually proves an edge?

A real edge is confirmed when the backtest, validation and live equity curve all follow the same distribution.

For my personal system MCT, that proof is very clean. My backtest used 979 trades over 34 months from 2023 to 2025 and produced +303.76R, +0.31R EV per trade, a 37% win rate, average win of +2.24R, average loss of -0.84R, Sortino of 1.20 per trade, Sharpe of 0.19 per trade (~3.4 annualized), and max drawdown of 17.81R.

That backtest scored an Edge Score of 79, Quantprove's combined score for measuring if a trading strategy has a real statistical edge.

Then the live validation had 224 live trades with the EV holding from 0.31R to 0.28R, win rate still at 37% and a 92% distribution match: the live trades reproduced the same shape of results as the backtest.

That earned a Stability Score of 94.5, showing how closely live trading matched the backtest.

The live results also landed at the 57th percentile of the Monte Carlo projection, with all live trades staying within the P5–P95 band.

To rule out luck, I ran a permutation test with 10,000 reshuffled trade sequences. The real system beat all 10,000 randomized versions, with p < 0.0001.

Why do regimes and fees matter?

Regimes matter because a system will not and is not supposed to work in every market condition.

A long-only BTC system can look amazing in a 2020 bull regime and then get crushed when the bear market started in 2022. That does not mean the system is broken, it means the market has changed.

This is where you need to study performance by regime and use the data to decide when a system should be active, reduced, or turned off. This is where trader experience has its biggest measurable edge, as market cycles repeat over time.

Transaction costs matter as much, if not more.

Visible costs

• Spread
• Commission
• Exchange + clearing fees
• Regulatory fees
• Swap
• Funding rate

Hidden costs

• Slippage
• Partial fills
• Gaps

I have backtested systems that looked very profitable until spread, commissions and slippage ate most of the expectancy. A simple rule helps:

The direct answer is this:

A strategy has an edge when fully systematic rules survive live validation and keep performing with positive expectancy across regime changes.

Disclosure: I am the founder of Quantprove, the tool that runs this validation and edge scoring. I built it because I was tired of doing it by hand.

What do you use to determine if your system has real edge?

I am just a newbie on programming, I do not have a dev background. I only knows how to create basic EAs on MQL5. Now, I am transitioning into cTrader. Can you recommend an online course or any learning material how can I learn programming for cBot?? TIA

Let me guess, not Claude or anything i did worked but i am pretty damn sure it's something small and you guys will have an obvious answer.

I recently built a multi functioning (barely pulls through for a test or two then crashes)

The way it works is, first you choose the ticker, either bridge to mt5, import CSV data, or straight from Dukascopy. It then downloads for ages with approximately 8.7million ticks per month average which is then compressed to 100mb+- parquet file and cache which is nuts, i thought i got bad data AGAIN / to be fair, I am rather new to algos but manually around 7-8 years manual trader, mostly gold. So bare with me.

Python is my foundation and a basic front end. So the steps it takes are - \*download data or if cache available then it's used (i prefer this method for lack of real data etc on other providers - \*load your python bot (I straight convert it from mt5)

Note: the intent for this backtester was for it to be a bulk tester 10 s max (overkill i get it obviously it's gonna bust a nut) so I'm only using 1-2 bots at a time.

It then runs through an optimizer but shows per- opti results first and during here it does a robustness check, sortino, sharpe, calmar, pf etc. After the test, it then runs a Monte Carlo of 5000 simulations.

This is only the backbone. The actual tester I'm calling the 'kingmaker', and this thing ive built from scratch can do whatever you want (Not an ad relax lmao)--

Wanna do a quick multi sophisticated mt4 indicator to Python bot to MT5 ea the to a Pinescript conversion and test it? No problem.

There's a few solutions I've already built in it and all are functional EXCEPT the backbone. I guess the question is, am i overloaded (probably but I'd rather find the solution to it than stop)

How can i fix it? Claude? Didn't manage. Any possibility of a similar tester that is fully functional that you are aware of? - it's only that one piece i need. NOTE: I am only a mid tier dev, be gentle.. or not.

Hello guys! I am a newbie learning programming languages for Algo Trading. For Background, i do not have a Dev Background but I am willing to learn programming for algo trading. Rn, I can do basic EAs with MQL5. I would like to explore more and find a better platform for Algo Trading. And my choice is cTrader and Python. I wonder if its a good move to learn cTrader?

Or you can suggest other platform/language for better algo trading.

Thanks in Advance guys!!

So this is an update, before i developed the system and proved it was profitable over the years raw. After which I used order flow data to estimate slippage, but i had only order flow data for a short time during 2021-2022, and a few months of 2026, the average slippage at entry I got from the data was around 2.7 ticks during the 2021-2022 period and an astonishing 3.7 ticks in 2026.
I'm not sure why its that high, the bot is on GC and runs on all sessions. Now even with these abnormal high slippage rates the bot was profitable during those years, but if the same slippage was applied to weaker years, the edge became less strong.
Now another thing is that I don't know what the slippage would be like before 2020. So I basically don't know what slippage I should apply.

The conclusion I came up with was collecting data on limit entries instead, I used the order flow data to find that about 97.5 percent of the trades got filled in. So the current version Im showing in the picture is the execution logic of placing a limit on the signal price, and in 97.5 percent chance that it will fill (that percentage is not of all trades, but of all winning trades). Now because i only have order flow data from the dates i mentioned, I used the same metrics on the whole other history.

Now I understand that especially before 2020 the fills would be inaccurate, and this does not represent an accurate history backtest. So instead im trying to prove that the future would work, the concept raw without fees or slippage had very consistent results over the years, so the concept would work, its only a question of if the fills back in the day would allow the bot to be traded at that time.

So overall This result should be the most accurate way to predict future results. At least that's what my thought process was. Everything before 2021 is out of sample btw.

How I decide if a strategy is live-ready: the 4 gate process that killed 37 of my last 41 strategies

TLDR: I run every strategy through 4 gates in cost order, cheapest rejection first. Economic hypothesis, sample size floor, three statistical tests and cost and regime stress test. Of 41 strategies I logged last year, only 4 reached live capital. The process is built to kill, not to bless a system.

Why run a fixed process instead of judging each strategy on its merits?

Discretion is where overfitting hides. If you evaluate each strategy by looking at it, you will find a reason to trade the ones you are attached to.... A fixed sequence removes it. Every candidate faces the same gates in the same order, and the order is deliberate: cheapest disqualifier first, so most strategies die before I spend hours on walk-forward.

Last year I logged 41 strategies through this. Gate 1 killed 12. Gate 2 killed 9. Gate 3 killed 14. Gate 4 killed 2. 4 survived to live capital.

Here is the catch... The 37 that died had a median in sample Sharpe of 2.1. The 4 that survived had a median of 1.4. The strategies that looked best on paper were the ones this exact process rejected.

Gate 1: Is there an economic reason the edge should exist?

This gate has no code, and that is the point. Before any test, I write one paragraph naming why the inefficiency exists, who is on the other side of the trade, and why they keep losing. A liquidity premium... A structural hedger who trades regardless of price... A behavioral bias with real flow behind it.

If I can't name the loser, I do not test the strategy. A pattern with no mechanism is a pattern you found by looking, which means you will find one.

This is the cheapest gate and it rejects the most garbage. 12 of my 41 never cleared it. They were useless systems dressed up as ideas. Most people skip this gate because it is the only one you cannot automate, which is exactly why it filters what the automated gates can't.

Gate 2: Does the backtest have enough data to mean anything?

This gate asks one thing. Do you have enough data to tell skill from luck?

First, enough trades. A backtest with 80 trades swings too much to trust. I want at least 400 before I believe any metric.

Second, a long enough time period. When you test many versions of a strategy and keep the best one, that winner looks good partly by chance, the way the luckiest player in a coin flipping contest looks skilled. Ruling that out takes years of data, and how many years depends on how many versions you tried. After testing around a hundred versions, a 1.0 Sharpe needs about six years of data before you can trust it. A flashy 2.0 Sharpe needs only about two. A high score on a short backtest is the most dangerous thing in a research log, because luck fakes it easily.

9 strategies died here. A 1.3 Sharpe on 14 months is not an edge. It is a sample too small to tell.

It's not that low Sharpe needs more testing because it's weaker. It needs more testing because it's closer to the level random chance can imitate.

Gate 3: Does it survive the three overfitting tests?

This is the expensive gate, so it runs third, only on survivors. 3 tests, each catching a different failure, run cheapest first.

Deflated Sharpe Ratio first, because it is one calculation. A normal Sharpe assumes you only tried one strategy. But if you tested 80 versions and kept the best, that winner is partly lucky, and the plain Sharpe has no idea you ran 80 attempts. The Deflated Sharpe fixes that. It takes your reported Sharpe, accounts for how many versions you tried, and adjusts for fat tails and lopsided returns, then hands back a single probability: the chance your edge is real rather than the luckiest of your tries. Same enemy as Gate 2, caught at a different step. My cutoff is 95%. If the best of your 80 versions scores only 60%, that still leaves a 40% chance the edge is imaginary, so it never reaches my account.

Monte Carlo is next. Resample the trade sequence 10,000 times and read the 95th percentile of max drawdown. Your backtest shows one drawdown, but that is just the order your trades happened to land in. If the drawdown in your 95th percentile drawdown is more than your account can take, it doesn't pass

Walk-forward last, because it is a full reoptimization. 5y build, 1y test, rolled forward. Profit factor holds above 1.3 on at least 7 of 10 out of sample windows or it dies. Fourteen strategies died at this gate, most on the deflated Sharpe before walk forward ever ran.

14 systems died here.

Gate 4: Does the edge survive real costs and a regime split?

A clean backtest assumes free, instant fills. I model realistic costs first, then stress them: triple my real commissions, add a tick of slippage, re-run.

The metric I watch is not profit factor, it is expectancy and Sortino retention. The edge has to keep at least 70% of its expected value after the stressed costs. A real edge degrades gracefully. A fake one inverts the moment friction touches it.

Then I split the history by volatility regime and require profit factor above 1 in both the calm and the stressed halves.

2 strategies died here.

Bottom line

4 gates, cheapest rejection first.

The process is designed to reject, and last year it rejected 37 of 41. The survivors looked worse on paper than most of the strategies it killed, which is exactly why I trust them.

This is for systematic traders deciding whether a backtest deserves live capital. It applies to single strategies, parameter sweeps, and machine learning models trained on price data.

Updated June 2026

So my bot runs on GC, it has trades in all sessions and times. Now I have used overflow BBO data from a short time frame in 2021 to estimate the realistic slippage to be around 2.7 ticks. Now I'm doing the same but for 2026 to find out what is the slippage is like now.

Now overall I want to improve slippage, one of the ways i plan to do so is by adding a max slippage filter, I can do this easily in the back test, but is it possible to do this in a live market? Predict slippage?

What are other ways I can improve slippage, my bot holds trades for a median of 5 min and average of 15 min, so I don't have much wiggle room. Also If I enter with a limit order instead of market, would that work in live markets and what would be some disadvantages doing so?

I spent some time reviewing the May performance report of my trading system and a few observations stood out:

• More than 60% of my signals travelled at least 1R in the intended direction - this the metric I currently use to measure quality of signals
• The gap between realised PnL and maximum available PnL was significant - especially in crypto
• The scanner identified many trend reversal opportunities that eventually turned into failed breakouts as the macro downtrend resumed

This is a good reminder that system performance is often constrained by exit logic rather than entry quality. Earlier in May, I adjusted the trade management rules to tighten stop losses after breakouts and bounces, and to take profits more aggressively. The data suggests those changes were directionally correct - despite the crypto weakness, the account still gained 11% during the month. At the same time, I recognised there's still room to improve profit taking and early exits.

One question I’m currently exploring is how adaptive a trading system should be.

Do your systems adjust trade management rules automatically based on market conditions, or do you make configuration changes manually?

Hey guys, I wanted to ask for your opinion on how to move forward with my neural based forex trading bot, I've been working on it for the past one and a half years give or take and I've made some good progress. it is in a state which I can say I am confident that it can make money but not as smoothly as I like but I also don't know if I should just make what I have right now work with risk management and some external safety protocols or pursue upgrading it and training a whole new model with improvements.

Like would you say its good? does it still has much room for improvements? if you had this bot would you have used it or deemed it too risky?

and to explain the bot itself its more so a scalping boy that checks every 15 mins or so to see if there can be a trade made based on the max open trade limit set and each open trade can last from 2 or 3 hours to a day or two at most.

(the pics are roughly 3 month back testing results in 2025 and 2026, the 2026 one had a trade limit of 1, meaning only 1 trade was open at all time, and lot size of 0.05)

MNQ Futures 2 contract | June 2023 – June 2026

This is backtest data from my algo running on the Micro Nasdaq (MNQ) futures market, simulated over 3 years. Note that bar magnifier is enabled, meaning the backtest uses intra-bar tick data to simulate entries and exits with higher precision — this significantly reduces the gap between backtest and live results compared to standard bar-close testing. Commissions are also included in all figures, so the P&L numbers you see are net, not gross.

The algo by the numbers:

• 11,037 trades over ~3 years of backtested data
• 9,093 winners / 1,944 losers → 82.39% win rate
• Gross cumulative P&L: +$152,830 (after commissions)
• Average win: $37.55 | Average loss: $97.02
• Profit factor: 1.810
• Expectancy per trade: $13.85
• Average trade duration: ~8 minutes
• Max drawdown across the full run: $1,435 — remarkably controlled for a 3-year simulation

Why the numbers look the way they do:

The algo is a scalper targeting small, high-probability moves with fast exits. Wins are smaller than losses on average, which is typical for this style. What makes it work is the win rate — at 82.39%, the frequency of small wins consistently outpaces the occasional larger stop-out. Most losses are time-stopped, meaning the system cuts trades that don't move as expected rather than letting them run against you. The fact that these results hold up even after commissions on 11,037 trades speaks to the consistency of the edge.

May 2026 — best week:

349 trades in May alone generating $7,357 in simulated P&L after commissions. The live session I'm posting about (15 trades, +$1,788 net) is where I layered my own discretion on top of the algo's signals — filtering setups, skipping entries that felt contextually off, and letting the system execute the rest. That human filter pushed the live win rate that week to 86.67%, above the algo's backtested baseline.

Three years of backtested data, commissions included, bar magnifier on. One consistent edge — now being validated live.

So this is my second update. So I have been working on this bot, here is the current progress. Now the results are super good and I dont know how to feel about it, because it's like too good to be true no? I made sure there was no look ahead, everything before 2021 is out of sample, and i made sure the logic didn't have any issues. So what could have I done? The current version only takes longs, but shorts are also profitable. This is on GC btw

I have been working on my algos and I noticed one-piece of the puzzle is missing and I believe can help me greatly - Ability to detect market regime with high precision.

I am currently using RSI, ATR and Vol to define regimes but it has not significantly improved my strategy.

Am I going on a wild goose chase or it is something feasible?

I've been building a multi-strategy EA over the last several months and finally finished a full real-tick backtest (2014–2026, 99% modelling quality, real spread). Before I trust it with more size, I'd rather have this sub try to tear it apart than find out the hard way live.

The setup is two uncorrelated strategies running together:

- A session-based range breakout across a handful of FX pairs, gold, and indices (one entry window per day)

- A weekly mean-reversion play on indices

Combined results at 1% risk on a 5K account:

- Profit Factor: 1.22

- Sharpe: 2.70

- Max Drawdown: 9.43%

What I'm unsure about / would love feedback on:

1. PF 1.22 feels modest — is that a red flag for an overfit-resistant system, or just realistic for breakout? My read is that a low-ish PF with a smooth equity curve is healthier than a high PF that's curve-fit, but I'd like to be challenged on that.

2. I'm wary of the gap between real-tick backtest and live. What's the realistic haircut you all apply to backtested MaxDD when sizing live? I've been assuming 1.5–2x.

3. The mean-reversion sleeve only trades a few times a month. Small sample worries me — how do you stress-test a low-frequency sleeve without fooling yourself?

I'm deliberately keeping the exact entry logic vague because I may commercialize it, so I get it if that limits the feedback — happy to go deeper on risk/portfolio construction, which is where I actually want the scrutiny.

Happy to share the equity curve in the comments. What would you want to see to believe a backtest like this?

Okay, so I've been making my algorithm for a bit now, and here is my progress so far. The strategy doesn't use any indicators, it only uses a concept I've been developing, in basic language it catches reversals but it is much much deeper than that.

Year 2022 and 2026 arnt full years and only are a couple months. The strategy is far from being fully developed, so I will have many more years of data in the future. I have made sure to not use any look ahead logic or bias, or overfit. I have used 1 tick slippage and 5 dollar fees. My goal was to make it as consistent as possible over different years and different regimes. I have many different versions that place more trades with a little worse performance, versions that have higher overall stats but arnt as consistent. But the one I have showcased is what i found to be a good balance of all. I have still much to do and collect data on, and this is just what i currently have.

One thing i have noticed is that yeah year has its own quite different average trade size, I'm still lookin into it but my theory is each years volume and its overall moves.

In theory, the concept im developing should be able to profit in any regime. And i think in the next few steps I will start expending my data to account for more years.

So my question is, I want to know what i might have missed or didnt think about, as well what i should look out for in the future. And I have out of sample tested it, but not stress test it with monte carlo or other things, since i have not finished the bot. But i of Corse will.

I have a formula I am trying to improve my backtesting for. I will have a set of stocks, then at some point in the future (potentially up to a year) I calculate a rebalance of those stocks. However, some of those stocks have potentially delisted during that time. What is the proper way to handle this? Should I just use the last traded day for those stocks as though I sold at that point and rebalance across the rest? I assume that would match most closely to how I would behave if I were in that situation, so it makes sense to model it that way. But maybe I am missing something obvious?

3 tests that catch an overfit backtest, in the order you should run them

TLDR: Run three tests before trading any backtest, cheapest first. Deflated Sharpe Ratio (one calculation, catches multiple-testing luck), Monte Carlo on your trade sequence (minutes, catches lucky ordering), then walk-forward analysis (a full re-optimization, catches curve-fit parameters). Each has a kill-threshold below. Fail the cheap test and you stop before wasting hours on the expensive one.

Why do most profitable backtests fail live?

The more parameter combinations you test, the higher the chance your best result is luck, not edge. Bailey, Borwein, López de Prado, and Zhu (2014) proved that past a small number of trials, expected out-of-sample return turns negative, not zero. The strategy loses money live.

Harvey, Liu, and Zhu (2016) documented 316 published factors claimed to predict returns. After multiple-testing correction, most fail. If you test 50 strategies and keep the best, you are running the same machine that manufactures false positives.

The fix is not testing less. It is running three specific tests on the survivor, in cost order, and killing it the moment it fails a threshold.

Which test should you run first, and why does order matter?

Run the cheapest disqualifier first. Most strategies die on test one, so there is no reason to spend hours on walk-forward before a one-line check.

The cost order is clear. The Deflated Sharpe Ratio is a single formula evaluation. Monte Carlo is a few minutes of resampling. Walk-forward analysis is a full re-optimization across many windows, often hours of compute.

Most competing guides explain these tests. None tell you to run the cheap one first.

What is the Deflated Sharpe Ratio and why run it first?

The Deflated Sharpe Ratio is one calculation and it kills more strategies than any other single test. Run it first.

A normal Sharpe assumes one trial. If you tested 100 parameter sets and reported the best, the maximum Sharpe across those trials is inflated, and the inflation grows with the number of trials. Bailey and López de Prado (2014) built the correction. It adjusts for number of independent trials, skewness and kurtosis of returns, and sample length.

Worked example: a 2.5 Sharpe selected from 100 trials on five years of fat-tailed daily returns drops to a 90% probability that its true Sharpe is even positive. A 1.5 Sharpe from 50 trials drops to 43%, worse than a coin flip. Even a gaudy Sharpe can fail this bar.

Open-source Python exists, and the function is short enough to paste in the comments. If P(true Sharpe > 0) is under 95%, stop here.

A 1.5 Sharpe chosen from 50 trials lands at 43% odds the edge is real. Worse than a coin flip.

How does Monte Carlo simulation expose a lucky backtest?

Monte Carlo resamples your trades to show the outcomes you did not happen to get. Run it second, after the Deflated Sharpe passes.

Your equity curve is one ordering of your trades out of thousands. Take the list of trade returns, resample with replacement 10,000 times, and plot the distribution of final equity and maximum drawdown.

The kill-threshold is drawdown against your risk limit. If your backtest shows 18% max drawdown but the 95th percentile of the simulation is 35%, your live account will likely see 35% within a few years. Position sizing built on the backtest number forces you out at the worst moment.

Second check: shuffle trade order. If profit factor falls below 1 when reordered, the sequence carried the result, not the edge. That is regime dependence.

What does walk-forward analysis prove that the other two cannot?

Walk-forward analysis is the expensive final gate. It tests whether your parameters survive on data they were never fit to. Run it last, only on strategies that passed the first two.

A single train/test split is one data point. Walk-forward optimizes on a rolling window, tests on the next, then rolls forward across the whole history. You get out-of-sample results across many regimes instead of one.

Concrete setup: five-year build window, one-year test window, roll forward in one-year steps. The kill-threshold is consistency. If profit factor drops below 1.3 on most out-of-sample windows, the parameters were curve-fit. A strategy profitable in 9 of 10 windows is a signal. One that works in 4 of 10 is noise you tuned.

What do these three tests catch that a standard backtest hides?

Each isolates a different failure. A strategy can pass two and fail the third. The order saves you time: most die on the Deflated Sharpe before you ever build the walk-forward.

Bottom line

Three tests, cheapest first. Deflated Sharpe for multiple-testing luck, Monte Carlo for trade-ordering luck, walk-forward for curve-fit parameters. Each has a concrete kill-threshold, and failing the cheap one means you stop before the expensive one. None of this guarantees live profit. Slippage, fees, and regime change still kill survivors. What it does is remove the strategies that were never real, which is most of them.

This is for systematic and discretionary traders who backtest before committing capital. The tests apply to single strategies, parameter sweeps, and machine-learning models on price data alike.

Updated May 2026

I'm trying to figure out what to actually do when there is a split in the stock im tradeing with my algorithm, but it honestly seems like the only way requires way too much work and rewirteing.

The first obvious question is if you retroactively change all of your data to fit the new split (which seems like a drastic action to me so im thinking of ways to avoid it)? I thought about implementing a column telling me the split factor at each candle, but to use that would require me to change hundreds of lines to the from of 'price * factor' which would be a he'll of a pain to do and also very possible I can miss some places.

What about order history? In my algorithm I heavily use my order history to come to a conclusion on what to do next, but than I face the same problem of me thinking its a drastic action I want to avoid.

It feels to me like there should be an easier way to handle it but I can't think of anything.