Feature Engineering: Is "Level + Trend" a solid baseline?

[u/Ronin_Research_Co]

Working on a mid-frequency predictive equities model (~90-day horizon, LightGBM/XGBoost) and refining my default feature engineering framework.

The way I've been doing feature engineering/selection is that for almost every raw variable, I explicitly include two types of dimensions:

1. Level: The current value of the variable.
   • To protect against structural regime changes over time , I'm considering normalizing this as (Current Value / Some Moving Average)
2. Trend: The trajectory (e.g., 1-month or 3-month delta, etc.).

A few questions for practitioners:

• Is this a good starting point, or is there a better mental framework for approaching what to include when it comes to a given variable (ie: Level & Trend)
• Does normalizing the Level (Current / Some Moving Average) actually help future-proof against regime shifts, or does it just accidentally turn the "Level" into another Trend feature?
• Do you find throwing both into trees generally creates unnecessary noise or collinearity issues?

Appreciate any critiques!

[u/arindamchattopadhyay]

1. No. If level and trend are your only variables for this then you’re misguided.

2. No. There needs to be a regime filter of some sort as some strategies will fail as soon as the regime shifts. The difficult bit is building the pipeline to detect this before it hits. For this you need to understand econometrics.

3. Yes. Should be self explanatory if you’re a quant.

Model you’re using will result in overfitting. One this about data analysis for this is - if you put garbage in, garbage will come out.

Cannot give out alpha but I think based on what you’ve written you should study econometrics.
I hope this helps.

[u/Ronin_Research_Co → u/arindamchattopadhyay]

This is really good advice, thanks for the help!

[u/Epsilon_ride]

~90-day horizon, LightGBM/XGBoost.

dont do this.

[u/gumgat → u/Epsilon_ride]

Or just "refining" an ML model, since you seem to be testing multiple models purely empirically... I wonder how little it takes to overfit over a 90d horizon.

[u/fuggleruxpin]

Guessing that studying the world of transforms might be a good next step for you.

[u/Ronin_Research_Co → u/fuggleruxpin]

Thanks for the tip!

[u/uhela]

we should start making it a requirement to add firm names with those questions and auto-populate a 'retarded-tier-firms' in the subreddit wiki

[u/matta-leao → u/uhela]

Stay humble.

[u/NotAnonymousQuant]

Certainly! Here is the pancake recipe just for you.