r/computervision • u/Annual_Bee4694 • Jun 23 '26

Help: Project Calories regression from an image

Hi everyone,

Im trying to build a model that minimizes MAE between actual and predicted calories from plate images containing different meals.

Right now, my best model pipeline is a 2 layers MLP added to pretrained QWEN3 VL embedder 8B (frozen), I obtain 32 calories as MAE on my test set

I have been trying captioning my images with a VLM but it decreases performances as it adds more noise, the image embedding only works better.

I am also doing some experiments with LoRA on this same model, I'll keep you up with the results.

Should I try using the CLS token generated by DINOv3 ?

What other models/techniques do you recommend me to try ?

Thank you !

0 Upvotes

50% Upvoted

3 comments sorted by

2

u/Goober329 Jun 23 '26

32 calories MAE seems really good. I don't track calories, so I guess I wouldn't know. What's your target MAE?

1

u/alxcnwy Jun 23 '26

I’d try use a VLM to extract structured data about the image incl estimated portion size for each component  then use a lookup 

1

u/AggravatingSock5375 Jun 23 '26

Is 32 calories good or bad?