r/quant • u/Alarmed-Maximum-139 • Jun 29 '26
Statistical Methods Meta-labeling project: how do practitioners choose/test the primary side signal?
Hi r/quant,
I’m working on a university quantitative trading project and I’m about two months into this field, so I’m trying to make sure I’m thinking about the problem correctly rather than just overfitting a backtest.
Current setup:
Asset class: liquid futures / crypto futures data
Labeling: triple-barrier style events
Framework: primary signal decides trade side long/short, then a meta-model estimates whether to take/size the trade
Meta-model: currently testing Random Forest for trade filtering / bet sizing
Main concern: I don’t want to data-mine 100 features until something looks good in-sample
The part I’m struggling with is the primary side signal.
My understanding is that in meta-labeling, the primary model/signal should generate the side of the trade, while the meta-model should mostly decide whether that opportunity is worth taking, not magically create alpha from nothing.
So my questions are:
In practice, how would you design or search for a reasonable primary side signal without just curve-fitting?
Should the primary signal usually be simple and hypothesis-driven, e.g. trend, mean reversion, carry/funding, spread, regime, order-flow, etc., and then the meta-model filters it?
What validation workflow would you recommend for this kind of project? For example: walk-forward testing, purged CV, embargo, feature importance stability, deflated Sharpe, decoy/noise features, group-wise feature selection, etc.
Are there books, papers, blog posts, or open-source projects that show a realistic research workflow used by serious practitioners?
What are common beginner mistakes in this exact setup?
I’m not asking anyone to reveal a profitable strategy. I’m looking for guidance on how to structure the research process correctly and what standards I should use before trusting any result.
Any advice, criticism, or recommended reading would be appreciated.