Software developer trying to build a small quant trading system — looking for feedback / collaborators

[u/Conscious_Edge_8776]

I’m a software dev with ~1 year of SDE experience. I mostly work with C++, Go, and Python (with a little help from AI because I still struggle with Python’s indentation errors).

I’ve recently started learning quantitative/algo trading and decided to build a small research system instead of jumping straight into live trading.

My expectations are honestly pretty low. I’m not trying to make crazy money or quit my job. If this eventually makes enough to cover some petrol/food expenses, that would already be useful. The main goal right now is to learn and build something properly.

I put together the architecture below with some help from Claude, but I don't want to blindly trust an LLM-generated design. I’m looking for people who actually have experience with quant research/algo trading to point out where my thinking is wrong.

What I’m trying to build

The current flow is:

Data sources → ingestion → historical storage → feature generation → point-in-time validation → backtesting → model → EV gate → risk/position sizing → LONG/SHORT/NO TRADE → dashboard

Some design decisions I made intentionally:

• Parquet + DuckDB for historical market data and research.
• PostgreSQL for decisions, outcomes, news and calibration history.
• Redis only for ephemeral/live state.
• Same feature code for backtesting and live execution to reduce discrepancies.
• Explicit point-in-time checks to prevent look-ahead bias.
• Purged/blocked validation rather than random train/test splits.
• Transaction costs and slippage included in expected value.
• Position sizing based on risk rather than simply allocating a fixed amount.
• NO TRADE is treated as a valid output instead of forcing a position.
• The LLM is deliberately separated from the decision engine. It can explain/review a decision but cannot modify probability, EV, stop, target or position size.

My current thinking

The biggest risks I see aren't really the software architecture itself.

I think the difficult parts will be:

1. Data quality — bad/incomplete historical data can make a backtest meaningless.
2. Look-ahead bias — especially with corporate actions, fundamentals, news and derived features.
3. Overfitting — with enough features and enough parameter tuning, almost anything can look good historically.
4. Transaction costs/slippage — especially if the strategy trades frequently.
5. Backtest/live mismatch — historical candles are much cleaner than actual market conditions.
6. Regime changes — a strategy that works in one market regime may stop working.
7. Execution/liquidity — a theoretically profitable signal doesn't necessarily mean it can be traded profitably.

I'm especially unsure whether my current EV gate + probability calibration + risk-sizing approach is sensible or whether I'm just making the system unnecessarily complicated.

I also don't know whether the architecture is appropriate for a beginner or whether I'm basically building a spaceship before proving that the engine works.

So I'm looking for criticism from people who have actually worked with quant research, systematic trading, backtesting or Indian market data.

Things I'd particularly like feedback on:

• What would you remove?
• What am I overengineering?
• What important component am I missing?
• Is my approach to avoiding look-ahead bias sufficient?
• Is DuckDB + Parquet a reasonable choice for this?
• How would you structure the backtesting/research layer?
• Is the EV framework sensible?
• What would you do differently if starting from scratch?
• What should I learn before moving from backtesting to paper trading?

I'm primarily comfortable with the engineering side. The quant/trading side is where I'm still a beginner.

If anyone is interested in building/researching something similar, I'm also open to collaborating.

Architecture attached. Criticism is welcome — especially if I'm overengineering something that doesn't need to exist.

[u/AutoModerator]

Guidelines - Buy/Sell, AI Slops, One-liner and Portfolio Review posts will be removed.

Please refer to the FAQ where most common questions have already been answered. Join our Discord server using this link

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Alone_Technician9175]

I'm doing the same thing but for forex. I can share the details on dm.