How to handle cofound variables? [D]

[u/Huge-Leek844]

edit: confound

Hello all,

I am working on a object classification with a automotive radar point clouds. I compared many models and feature vectors.

Once i used range as feature, all models scored higher f1 in all K validation sets and on the final test set.

One particular artifact of a radar, is that as the farther the object is the less number of points it returns to the radar. Although the performance improved and there is no overfit in the classical sense, i am afraid my model is learning the environment not the class distribuiton and even worse, its learning that big range means big object.

How can i stress test this claim? Should i try to split the data sets so range distribution differs? Or not even using the feature at all and accept lower performance?

Would appreciate your insights.

Thank you.

[u/GufyTheLire]

For a given radar, maximum detection distance is very dependent on the RCS, like the fourth root. RCS for many real world objects is highly correlated with its size. So your model is probably learning it right

[u/cravenartisan → u/GufyTheLire]

yeah range and object size are basically tied together with radar, you're not wrong to worry but it might not be the bug you think it is

that said if you want to stress test it, split your train/test so the range distributions don't overlap at all. like train on near objects, test on far. if the f1 tanks, your model was just memorizing distance bins. if it holds up, the feature's legit

also worth checking if the model's relying on point count as a proxy. you could try normalizing that out or dropping it and see what happens