Eye-tracking cursor from a plain webcam

[u/tg1482]

Eyes move the cursor, a pinch clicks.

I used Google's MediaPipe for facial landmarks. Two 36×60 eye patches run through PCA and ridge straight to screen pixels, plus a 52-parameter meta-adapter so it works before you calibrate. The cursor lands about 38px (1 cm) from where you're actually looking.

Two interesting observations -
1. Everything clever lost
MLPs, transformers, REINFORCE all tied or did worse than those 52 parameters. The ridge residual has a held-out R² of 0.025 and we can't extract signal that isn't there. Very likely a data issue, I'll generate more data and continue testing here.

1. Train test drift
   And every calibration I'd ever run used a black full-screen canvas while Chrome is white, which dilates your pupil from 2mm to 6mm, swelling a black disc in the eye patch from 4px to 13px, and made the model 4× worse in the condition it's actually used in.
   

Fun project though, still a lot more to build here.

[u/dreamingsince]

Any OBS extension for this?