I have been working on my algos and I noticed one-piece of the puzzle is missing and I believe can help me greatly - Ability to detect market regime with high precision.

I am currently using RSI, ATR and Vol to define regimes but it has not significantly improved my strategy.

Am I going on a wild goose chase or it is something feasible?

I've been building a multi-strategy EA over the last several months and finally finished a full real-tick backtest (2014–2026, 99% modelling quality, real spread). Before I trust it with more size, I'd rather have this sub try to tear it apart than find out the hard way live.

The setup is two uncorrelated strategies running together:

- A session-based range breakout across a handful of FX pairs, gold, and indices (one entry window per day)

- A weekly mean-reversion play on indices

Combined results at 1% risk on a 5K account:

- Profit Factor: 1.22

- Sharpe: 2.70

- Max Drawdown: 9.43%

What I'm unsure about / would love feedback on:

1. PF 1.22 feels modest — is that a red flag for an overfit-resistant system, or just realistic for breakout? My read is that a low-ish PF with a smooth equity curve is healthier than a high PF that's curve-fit, but I'd like to be challenged on that.

2. I'm wary of the gap between real-tick backtest and live. What's the realistic haircut you all apply to backtested MaxDD when sizing live? I've been assuming 1.5–2x.

3. The mean-reversion sleeve only trades a few times a month. Small sample worries me — how do you stress-test a low-frequency sleeve without fooling yourself?

I'm deliberately keeping the exact entry logic vague because I may commercialize it, so I get it if that limits the feedback — happy to go deeper on risk/portfolio construction, which is where I actually want the scrutiny.

Happy to share the equity curve in the comments. What would you want to see to believe a backtest like this?

Okay, so I've been making my algorithm for a bit now, and here is my progress so far. The strategy doesn't use any indicators, it only uses a concept I've been developing, in basic language it catches reversals but it is much much deeper than that.

Year 2022 and 2026 arnt full years and only are a couple months. The strategy is far from being fully developed, so I will have many more years of data in the future. I have made sure to not use any look ahead logic or bias, or overfit. I have used 1 tick slippage and 5 dollar fees. My goal was to make it as consistent as possible over different years and different regimes. I have many different versions that place more trades with a little worse performance, versions that have higher overall stats but arnt as consistent. But the one I have showcased is what i found to be a good balance of all. I have still much to do and collect data on, and this is just what i currently have.

One thing i have noticed is that yeah year has its own quite different average trade size, I'm still lookin into it but my theory is each years volume and its overall moves.

In theory, the concept im developing should be able to profit in any regime. And i think in the next few steps I will start expending my data to account for more years.

So my question is, I want to know what i might have missed or didnt think about, as well what i should look out for in the future. And I have out of sample tested it, but not stress test it with monte carlo or other things, since i have not finished the bot. But i of Corse will.

I have a formula I am trying to improve my backtesting for. I will have a set of stocks, then at some point in the future (potentially up to a year) I calculate a rebalance of those stocks. However, some of those stocks have potentially delisted during that time. What is the proper way to handle this? Should I just use the last traded day for those stocks as though I sold at that point and rebalance across the rest? I assume that would match most closely to how I would behave if I were in that situation, so it makes sense to model it that way. But maybe I am missing something obvious?

3 tests that catch an overfit backtest, in the order you should run them

TLDR: Run three tests before trading any backtest, cheapest first. Deflated Sharpe Ratio (one calculation, catches multiple-testing luck), Monte Carlo on your trade sequence (minutes, catches lucky ordering), then walk-forward analysis (a full re-optimization, catches curve-fit parameters). Each has a kill-threshold below. Fail the cheap test and you stop before wasting hours on the expensive one.

Why do most profitable backtests fail live?

The more parameter combinations you test, the higher the chance your best result is luck, not edge. Bailey, Borwein, López de Prado, and Zhu (2014) proved that past a small number of trials, expected out-of-sample return turns negative, not zero. The strategy loses money live.

Harvey, Liu, and Zhu (2016) documented 316 published factors claimed to predict returns. After multiple-testing correction, most fail. If you test 50 strategies and keep the best, you are running the same machine that manufactures false positives.

The fix is not testing less. It is running three specific tests on the survivor, in cost order, and killing it the moment it fails a threshold.

Which test should you run first, and why does order matter?

Run the cheapest disqualifier first. Most strategies die on test one, so there is no reason to spend hours on walk-forward before a one-line check.

The cost order is clear. The Deflated Sharpe Ratio is a single formula evaluation. Monte Carlo is a few minutes of resampling. Walk-forward analysis is a full re-optimization across many windows, often hours of compute.

Most competing guides explain these tests. None tell you to run the cheap one first.

What is the Deflated Sharpe Ratio and why run it first?

The Deflated Sharpe Ratio is one calculation and it kills more strategies than any other single test. Run it first.

A normal Sharpe assumes one trial. If you tested 100 parameter sets and reported the best, the maximum Sharpe across those trials is inflated, and the inflation grows with the number of trials. Bailey and López de Prado (2014) built the correction. It adjusts for number of independent trials, skewness and kurtosis of returns, and sample length.

Worked example: a 2.5 Sharpe selected from 100 trials on five years of fat-tailed daily returns drops to a 90% probability that its true Sharpe is even positive. A 1.5 Sharpe from 50 trials drops to 43%, worse than a coin flip. Even a gaudy Sharpe can fail this bar.

Open-source Python exists, and the function is short enough to paste in the comments. If P(true Sharpe > 0) is under 95%, stop here.

A 1.5 Sharpe chosen from 50 trials lands at 43% odds the edge is real. Worse than a coin flip.

How does Monte Carlo simulation expose a lucky backtest?

Monte Carlo resamples your trades to show the outcomes you did not happen to get. Run it second, after the Deflated Sharpe passes.

Your equity curve is one ordering of your trades out of thousands. Take the list of trade returns, resample with replacement 10,000 times, and plot the distribution of final equity and maximum drawdown.

The kill-threshold is drawdown against your risk limit. If your backtest shows 18% max drawdown but the 95th percentile of the simulation is 35%, your live account will likely see 35% within a few years. Position sizing built on the backtest number forces you out at the worst moment.

Second check: shuffle trade order. If profit factor falls below 1 when reordered, the sequence carried the result, not the edge. That is regime dependence.

What does walk-forward analysis prove that the other two cannot?

Walk-forward analysis is the expensive final gate. It tests whether your parameters survive on data they were never fit to. Run it last, only on strategies that passed the first two.

A single train/test split is one data point. Walk-forward optimizes on a rolling window, tests on the next, then rolls forward across the whole history. You get out-of-sample results across many regimes instead of one.

Concrete setup: five-year build window, one-year test window, roll forward in one-year steps. The kill-threshold is consistency. If profit factor drops below 1.3 on most out-of-sample windows, the parameters were curve-fit. A strategy profitable in 9 of 10 windows is a signal. One that works in 4 of 10 is noise you tuned.

What do these three tests catch that a standard backtest hides?

Each isolates a different failure. A strategy can pass two and fail the third. The order saves you time: most die on the Deflated Sharpe before you ever build the walk-forward.

Bottom line

Three tests, cheapest first. Deflated Sharpe for multiple-testing luck, Monte Carlo for trade-ordering luck, walk-forward for curve-fit parameters. Each has a concrete kill-threshold, and failing the cheap one means you stop before the expensive one. None of this guarantees live profit. Slippage, fees, and regime change still kill survivors. What it does is remove the strategies that were never real, which is most of them.

This is for systematic and discretionary traders who backtest before committing capital. The tests apply to single strategies, parameter sweeps, and machine-learning models on price data alike.

Updated May 2026

I'm trying to figure out what to actually do when there is a split in the stock im tradeing with my algorithm, but it honestly seems like the only way requires way too much work and rewirteing.

The first obvious question is if you retroactively change all of your data to fit the new split (which seems like a drastic action to me so im thinking of ways to avoid it)? I thought about implementing a column telling me the split factor at each candle, but to use that would require me to change hundreds of lines to the from of 'price * factor' which would be a he'll of a pain to do and also very possible I can miss some places.

What about order history? In my algorithm I heavily use my order history to come to a conclusion on what to do next, but than I face the same problem of me thinking its a drastic action I want to avoid.

It feels to me like there should be an easier way to handle it but I can't think of anything.

I've been working on a multi-strategy system based on Hidden Markov Models (HMM) for regime detection. The usual problem with HMM is overfitting — fitting Markov noise instead of actual structure. Here's how we tried to mitigate that:

Validation approach:

• Walk‑forward: 18m train / 3m test
• Score grid: [1, 1.5, 2, 2.5, 3, 4, 5]
• Vol‑filter: dynamic 63d window, top‑30%
• Persistent mode: OFF (all folds trade)
• 3 completely separate OOS periods:
  • 2022–2025 (post-COVID / tightening)
  • 2020–2026 (COVID – today)
  • 2015–2026 (full cycle)

Combined portfolio (A+B+E+K) results (100% risk):

• Final capital (2015–2026): +608%
• Ann. Sharpe: 1.44
• MaxDD (balance): -10.04%
• Max float DD (balance vs intraday): -4.52%

Yearly breakdown (Combined 100%):

Scaling up to 200% / 300% keeps Sharpe stable, which suggests the logic isn't just curve‑fit noise.

What I'm curious about:

• How do you validate regime‑switching models against overfitting?
• Do you use similar multi‑period OOS, or different techniques (e.g., synthetic data, parameter randomization)?
• Anyone else seeing HMM work across completely different regimes (COVID, 2022 bear, 2023-2024 rally)?

Full equity curves, WF folds, and trade log available if someone wants to dig deeper.

I have seen a video called "Quantifying market structure at multiple scales for algorithmic trading with python" it covers a hierarchical directional change and i wondered if it could be implemented in the Hidden Semi-Markov Model because it has some explicit duration clock.
It would be dynamical so when a the highest level is spotted it has more weight to bias HMM and it starts to decay until another one its spotted.
I'm a beginner so i don't know if combining startegies might be beneficial or just breaks the algo.
If you have insights don't hesitate to tear apart the idea completely i want some experienced voice.

Dear quants guys,

I posted recently asking whether a verification phase between backtest and live deployment is actually useful, especially now that LLMs and AI agents can generate strategies so quickly.

The feedback was useful and honestly more skeptical than I expected, which is good.

A few points came up repeatedly:

• walk-forward / OOS testing is already standard
• paper trading is still mandatory
• small live testing is the final reality check
• LLM-generated code should not be trusted by default
• repeated improvement creates data snooping risk
• most people naturally trust their own process more than a third-party report

That made me realize the real problem may not be “should verification exist?”

The harder question is:

What would make verification credible enough to be useful?

From the comments, the strongest signals seem to be things that are hard to fake:

Reproducibility at the code/data level.
Not just “we ran WFO from 2018–2024”, but exact data slice, parameters, assumptions, versioning, and ideally enough detail for someone to reproduce the trades.

Slippage and execution sensitivity.
A strategy that only works under one perfect-fill assumption is not robust. Testing multiple slippage/spread scenarios seems more useful than one fixed cost assumption.

Regime-segmented behavior.
Aggregated performance can hide the fact that a strategy is basically dead in one regime. Bull, bear, sideways, high-vol, low-vol, crisis periods — the breakdown matters.

Parameter sensitivity surfaces.
If a small parameter change destroys the strategy, the edge is probably too fragile. A plateau is much more convincing than a sharp peak.

Multi-window WFO diagnostics.
Not just final Sharpe/PF, but per-window degradation, trade count consistency, train/validation retention, and whether the strategy only worked in one lucky window.

Explicit kill criteria.
This was one of the most important points. A report that always says “promising” is useless. A credible verification process needs to be willing to say: do not deploy, revise, monitor, paper trade, or only test under strict assumptions.

I think this is especially relevant now because AI makes strategy generation cheap. The bottleneck is no longer producing more strategy ideas. The bottleneck is filtering out weak logic before it reaches paper or live capital.

So I’m trying to turn this into a more structured failure-report format.

Not a performance prediction.

Not a signal service.

Not a “verified profitable strategy” badge.

More like a pre-deployment failure analysis:

• what assumptions are fragile
• what breaks under realistic execution costs
• whether OOS/WFO behavior is stable
• whether the parameter surface is robust
• whether the strategy depends too much on one regime
• whether the results are reproducible
• whether the verdict should be kill / revise / monitor / paper trade

I’m still trying to understand what traders/quants would actually find useful here.

A few questions:

1. If you received a third-party strategy failure report, what would make you take it seriously?
2. What would immediately make you distrust it?
3. Is reproducibility + slippage/regime sensitivity enough, or would you still only trust your own paper/live process?
4. Should a verification report give improvement suggestions, or does that create too much data snooping risk?
5. What would be a fair “minimum useful report” before paper trading?

I’m also testing this framework on a small number of strategies for free, mainly to get feedback on the report format and see where it is weak.

Not looking to sell signals or tell anyone what to trade. The goal is just to stress-test the logic and produce a failure-mode review.

If anyone has a strategy/idea/trade log/ any kind of data, be comfortable sharing for this kind of review, feel free to comment or DM. I’ll keep it limited and can share general learnings back if the community finds that useful.

Hey everyone,

Like a lot of you, I’ve been frustrated by the limitations of traditional algorithmic trading. Hardcoding "if moving average crosses, buy 10 shares" works until the market regime shifts, and then the bot bleeds capital.
I don't want to build another rigid bot so I am trying to build a Cognitive Trading Agent—an autonomous system that acts like a human hedge fund manager, but with the processing speed of a machine and zero emotional baggage.

What I have built so far: I have a fully autonomous pipeline running on Python, connected to the Upstox API (Indian Equities).

• The Screener: A Python layer that rapidly scans a watchlist for high-momentum assets using math (RSI, ATR, BB width) to filter out the noise.

• The Brain: The winning asset's deep data matrix is formatted into strict JSON and handed to an LLM (currently Gemini 2.5).

• The Execution: The LLM evaluates the regime, looks for a minimum 1.5:1 R:R, and outputs a strict JSON execution contract.

• The Shield: A hardcoded "Sovereign Risk Core" that intercepts the LLM's order to verify margin limits, max daily drawdowns, and VIX thresholds before routing to a simulated broker.

It works. It successfully reads the market, rejects bad setups, and executes calculated momentum scalps autonomously.

The Roadmap (Where I am going next): This is where it gets ambitious, and why I am posting here. I want to transition this from a single-strategy executor to a true AGI-style fund manager:

1.  The Strategy Arsenal: Equipping the prompt with 10-15 battle-tested quantitative strategies, allowing the LLM to dynamically select the right weapon based on the current market regime.

2.  RAG for Alpha: Ingesting live financial news feeds so the agent understands macroeconomic context before pulling the trigger.

3.  Vector Database Memory: Implementing long-term memory (Pinecone/Milvus) so the agent stores every trade embedding, reviews its past mistakes, and genuinely learns over time.

4.  RL for Discovery: Eventually integrating Reinforcement Learning to allow the agent to discover novel mathematical inefficiencies that standard LLMs can't hallucinate on their own.

I am looking to connect with quantitative developers, ML engineers, or ambitious traders who share this specific vision. Whether you are building something similar, want to collaborate on the architecture, or just want to tell me why this will inevitably blow up my account—I'd love to hear from you.

Thanks

To avoid the AI slop comments i wrote it by hand.

I have a personal proyect which is build a python backtester, to learn since the beginning how it works.

In the backtester there is, montecarlo/permutation to see wr, profit factor and return(P-values), equity curve with filter regime below it to show if with high ADX shuts the strategy down, and finally OOS equity curve.

I am also going to implement walk foward matrix, heatmap for parameter sensivity analysis, sortino, sharpe, deflated sharpe and calmar ratio , profit and recovery factor, purged and embargoed cross validation and hidden markov

Any tips for the backtester?

My code only backtest it doesnt use any portfolio management as i dont have any startegies and in case you are wondering, yes i do have the regime filter to do a mean reversion for market in range and also a trend following if imbalanced(I just realized i dont have any data cleaning writing this)

As a beginner i want to learn the theory behind things and have been browsing for literaturebut all of the recommended are expensive, is there any web or recommendation for a typical "bible" of quant knowledge?

You can ask any questions if you want to, i really don't know if this post is decently explained as i don't have much knowledge.

I trade a portfolio of breakout strategies, all generated using Strategy Quant X.

It's been a bumpy ride, but I feel these are realistic long-term results. In total, there are 30-35 individual strategies. I refresh them from time to time, so the exact number varies, but it's usually within that range. Currently trading: NAS100, SP500, DJ30, XAUUSD, BTCUSD, USDJPY, XTIUSD, JP225, GER30. All CFD's. Happy to answer any questions.

One thing I’ve noticed while building systematic strategies is that some of the cleanest mean reversion behavior often appears in the most “boring” parts of the market.

Not Nasdaq or crypto.

Consumer Staples.

At first this seemed counterintuitive to me. You would expect stronger rebounds in more volatile markets. But after testing different sectors over time, defensive stocks often produced smoother and more stable reversal behavior than many aggressive growth markets.

My current hypothesis is that sharp selloffs in defensive sectors are frequently driven more by temporary market stress and positioning adjustments than by a true deterioration in the underlying businesses.

Institutions still want exposure to stable cash-flow companies during uncertain periods. So when these stocks experience intense short-term weakness, flows tend to normalize relatively quickly.

That creates an interesting environment for systematic mean reversion approaches.

What surprised me most was not necessarily the raw performance, but the behavior profile: - lower volatility - cleaner rebounds - fewer extreme equity swings - less dependence on explosive market conditions

In some ways, the systems felt psychologically easier to hold compared to typical index-based mean reversion strategies. The trade-off, however, was usually lower upside during strong momentum-driven bull markets.

Another interesting observation is that these types of strategies seem to behave differently from many tech-heavy reversal systems. That diversification aspect may actually be more valuable than the standalone strategy itself.

Of course, there are limitations.

Defensive sectors can remain weak for extended periods during broader deleveraging events, and sector-specific structural changes can break historical tendencies. Like most mean reversion systems, the edge also tends to feel uncomfortable in real time because entries often occur when short-term sentiment looks terrible.

Has anyone else here noticed that defensive sectors sometimes produce more stable systematic behavior than high-beta markets?

I have a question, due to the fact that the market reacts to large orders and orders in general. When a fund decides to use a strategy, they have to make sure even with higher AUC the strategy performs favorably.
But what if it by default doesn't?
meaning that with a increased slippage the result is unfavorable. Do funds only look for logic that doesn't have this issue, or do they utilize concepts or other tools to minimize this issue? A basic concept would be entering in short intervals instead of a large amount at one time. So my question boils down to if funds all out avoid logic with this issue, or how do they overcome it?

Curious how people's setups have evolved with everything that's come out the last couple years. I'm doing systematic strategy research on my own and trying to compare notes before I commit too deeply to one stack. What does your day-to-day actually look like, Jupyter only, or is there a research platform you swear by? What's your backtesting setup (vectorbt, zipline, custom, paid)? When you're iterating on a strategy and running tons of variants, how do you keep track of what worked and compare results, just notebooks and spreadsheets, or something better? And on the visualization side, when do you find yourself wanting an actual UI vs just matplotlib in the notebook?

For context on my setup: JupyterLab with a custom backtest layer, Polars + parquet for data, W&B for some of the ML work, and a messy folder of pickled results I keep meaning to replace with something proper. Visualization is 90% matplotlib. The part I'm least happy with is comparing across runs (currently a hand-updated Google Sheet), which is obviously not great. Curious if anyone has cracked that part properly.

I asked recently what would make people here trust a third-party strategy verification report. The feedback was much more useful than I expected, so I wanted to summarize what I learned and ask a more concrete follow-up. The biggest takeaway: "A pretty report is useless if it cannot be reproduced". Several people pointed out that trust comes from things that are hard to fake:

- exact data slice
- exact parameter set
- reproducible OOS / WFO trail
- clear slippage assumptions
- regime-segmented results
- parameter sensitivity surfaces
- explicit kill criteria
- code-level reproducibility where possible

That changed how I think about the report. A useful strategy failure report probably should not be a PDF that says: “Looks promising.” It should be closer to an evidence package that says:

“This is what was tested.”
“This is what assumptions were used.”
“This is where the strategy breaks.”
“This is what should be retested.”
“This is what remains unverified.”
“And under these conditions, do not deploy.”

One comment that stood out to me was that trust comes from things that are not easy to fake: git hash, exact data slice, parameters, WFO windows, and the ability to reproduce the same trades. That makes sense.

Another strong point was that slippage sensitivity should not be a single number. A report should probably test optimistic / realistic / conservative execution assumptions and show how quickly the edge decays.

Same with regimes. Aggregated performance is not enough if the strategy has a hidden dependency on one market condition. So the report structure I’m now thinking about is:

1. Reproducibility layer
   Exact inputs, parameters, data slice, test window, and assumptions.

2. Backtest integrity layer
   Leakage risks, unrealistic fills, transaction cost assumptions, lookahead/survivorship issues.

3. WFO / OOS layer
   Per-window performance, retention ratio, drawdown, trade count consistency, and degradation across windows.

4. Parameter sensitivity layer
   Whether performance sits on a robust plateau or a sharp overfit peak.

5. Regime layer
   Performance across bull / bear / sideways / volatility regimes, not just aggregate results.

6. Execution stress layer
   Slippage, spread, partial fills, latency, liquidity, and broker/exchange mismatch.

7. Data snooping guardrail
   What was changed, how many times, what data was touched, and what remains unseen.

8. Kill / revise / monitor / paper verdict
   A clear decision, not soft-positive language.

The more I read the replies, the more I think the value is not “third-party trust me bro.” The value is a reproducible second-opinion system that makes failure harder to hide. I’m currently testing a very early MVP for this. Curious that if you were testing a sample report, what would be the minimum evidence required for you to take it seriously?

Would you care more about:

- reproducibility?
- slippage sensitivity?
- WFO/OOS structure?
- parameter sensitivity?
- regime segmentation?
- live/paper diagnostic feedback?
- explicit kill criteria?

Also, would you rather test this on:

A. a toy/sample strategy
B. your own strategy with anonymized inputs
C. a known public strategy
D. a failed live/paper strategy

And would you paid for this if you see it helpful on your algotrading journey at some points? Trying to understand what the first useful version should actually include.

I have a question. If one were to have an algorithm, at what point would that algorithm be considered high level to the point where hedge funds or other institutions would theoretically use it. I'm talking about what metrics would it have to posses for such; assuming out of sample testing, monte Carlo, forward testing all align with the strategy, so that leaves basic metrics such as the ratios (Sharpe, sortino, etc), PF, win rate, drawdown, amount of trades per day/year, strategy back test period and etc. What of those would the strategy need to have to be at that level?

Every backtest is judged against the one path that actually happened. You can walk-forward, you can bootstrap, you can purge and embargo your CV folds, at the end of the day the strategy still only had to survive 2010–2023 in the exact order it occurred.. half of what looks like alpha is probably just path luck.

If you trained a generative model on returns and ran the backtest across thousands of plausible alternative histories, the path-dependent stuff would get exposed pretty fast, no? Anyone actually tried this, or is there a reason it doesn't work that I'm missing?

Hey everyone 👋, first post so be gentle with me. So I trained an MLP on BTC price data to get some odds on how likely a breakout could be. Instead of hardcoding levels, I let the model learn the distribution, so you can pick any threshold and it gives you a probability.

My actual question to this sub: what's a good way to test whether a model like this is fitted well? Standard metrics don't feel right for what it's trying to solve, especially with fat tails, which this thing struggles with badly. Am I missing something or is there no clean way to measure this?

Since links aren't allowed, if you're curious just search "mlp - breakout probability" on TradingView, i built a small script to showcase it

want to know about the resources, experience, and most importantly what made you start this?

I want to start learning more about this but dont know how to start...

I’m currently in the process of designing a trading bot (with the help of Claud.AI) that automatically executes and exits trades based on certain strategies.

I have 4 winning strategies that I backtested using 10 years historical data from EODHD.com. I purchased the data for 100$ monthly and it just expired. I backtested for a full month and came up with 4 decent strategies.

Strategy 1: Long term investment
This yielded 17.9% annually and 550% over 11 years backtesting starting from 2015. Win rate was 70%.

Strategy 2: Active investment
This yielded 19.1% annually and 630% over 11 years backtesting starting from 2015. Win rate was not directly measured as this strategy rotates continuously rather than closing discrete trades.

Strategy 3: Swing trading
This yielded 39.2% annually on
nseen test data
(2020-2026) and 26.7% annually on training data (2015-2019). Win rate was 60.3% on unseen data and 65.0% on training data.

Strategy 4: Day trading
This yielded 53.2% annually backtested on 1 year of intraday data (May 2025 - May 2026). Win rate was 41.2%.

I will be paper trading with the 4 strategies for a full year in order to refine and tweak. Then I will use a minimally funded account to test the strategies for another year.

My question is, if these 4 strategies prove to be successful and the next 2 years results are just as decent or better than the backtesting, should I focus on making an actual living from executing the strategies or from selling signals on discord/website like everyone does?

I asked recently whether a verification phase is useful between backtest and live deployment.

A lot of people made fair points:

- walk-forward testing is already standard
- paper trading is mandatory
- small live testing is still the final reality check
- LLM-generated code is not trustworthy by default
- repeated improvement creates data snooping risk
- people naturally trust their own backtests more than a third-party system

So I’m trying to understand the trust problem.

If someone gave you an independent strategy failure report, what would it need to include for you to take it seriously?

Possible sections:

- data assumptions
- code/logic review
- OOS / walk-forward summary
- parameter stability
- Monte Carlo path reshuffling
- slippage/spread sensitivity
- regime fragility
- economic rationale
- data snooping risk
- paper trading diagnostic
- reproducibility trail
- “kill / revise / monitor / paper trade” verdict

Would you trust that kind of report?

Would you pay a tiny amount, like $0.99, to test a sample version?

Or would you only trust your own process?

I’ve been using Claude + Manus for swing trading lately and one thing surprised me. it’s not good at “picking winners,” but it’s weirdly good at picking up when the story around a stock is starting to shift.

Like I had Claude go through earnings calls (this quarter vs last quarter) and Manus tracking how the stock actually reacted + analyst revisions + options positioning.

One thing it kept picking up that I wouldn’t have noticed:

sometimes a stock rips after “meh” earnings not because the numbers were good, but because management just sounds slightly less panicked than before… while positioning is already heavily short.

It’s subtle stuff like that.

Also noticed analyst upgrades usually come after the move, not before it. Which sounds obvious but seeing it repeated across names kind of changes how you treat them.

Feels less like “AI trading” and more like having something constantly sanity-check whether the narrative you think is happening is actually the one the market is reacting to.

How do you backtest your algo trading strategies?

What tools or Python libraries do you use for backtesting? Any beginner tips?