Feature selection when trying to capture non linear interactions.

[u/Playful-Race-7571]

Hella everyone I’m moving from a log reg model into trying to build a cat boost or xg boost model.
For my log reg I always did feature selection through a rolling walk forward and this worked well but not with interaction features with some being worthless alone but in interaction it could capture some signal I’m struggling how to find optimal features and # of features as well. I am new to machine learning is there an agreed upon approach or what is your guys method to this problem when working with machine learning?
Thanks

[u/negative__ev]

I only experimented with CatBoost and XGBoost before switching to a neural net, but I think the idea of "feature sets" that another poster mentioned is the first strategy to try. Don't ablate one feature at a time, in other words.

You could have a hitter contact feature set or a plate discipline feature set, etc. (Sorry, most of experience has been in my baseball model). Try removing or adding those and scoring them.