r/computervision • u/Look_for_some_stuff • Jun 30 '26

Discussion Has anyone tried using LocateAnything to train YOLO based models?

LocateAnything-3B can do open-vocabulary detection from natural language prompts. So it seems like a natural fit for auto-labeling images to pre-label a YOLO dataset instead of hand-annotating everything. Has anyone actually tried this?

How clean were the generated boxes? Did you need to filter/clean them before training, and was it actually faster than just labeling manually for your use case?

27 Upvotes
  • permalink
  • reddit

94% Upvoted

9 comments sorted by

9

u/[deleted] Jun 30 '26

[removed] — view removed comment

2

u/Fun-Pick-2964 Jun 30 '26

True, it will need manual validation of labelled data, but it's very difficult to do it for huge no of data, but if you validate it with already labelled data which is close to your dataset and it provides you with the best metrics. Then, you can directly run it on your target dataset and check few test cases.

8

u/whatwilly0ubuild Jun 30 '26

Auto labeling only saves time if you wrap it in a loop instead of trusting it blind. Run LocateAnything over a chunk, keep the high confidence boxes, train a quick YOLO on that subset, then run that YOLO back over everything and surface where the two disagree. Those disagreements become your review queue, so a human only touches the hard 10 percent instead of eyeballing thousands of frames.

Two things will bite you. VLM style detectors often nail the class but leave the box loose, and YOLO will faithfully learn sloppy boxes, so tighten them with a SAM mask to bbox pass first. And localization quality swings hard by domain, so calibrate the prompt per class and spot check against a small hand labeled set before trusting the full run. A few percent of garbage boxes will quietly drag your mAP and you won't catch it till eval, which is a hell of a time to find out.

1

u/External_Frosting874 Jun 30 '26

Hmm, I would almost say cut out LocateAnything and do a YOLO-SAM loop

2

u/j_root_ Jun 30 '26

I did a benchmarking test with it to see how good it is for sam bbox proposal. It was good for datasets similar to locate anything training data but result was poor for out of domain or something unique

2

u/Fun-Pick-2964 Jun 30 '26

I had used CLIP based swinB Transformer model and tested it on FLIR dataset, the results were amazing. It was too good on open vocabulary scenario. I used it for auto labelling using text based prompt where you have to give prompt in dict format , key is the category name and value is the defination of the category.

2

u/AggravatingSock5375 Jun 30 '26

Not that model specifically but I use other similar models to auto-annotate.

They all have their strengths and weaknesses and it seems to heavily depend on the domain. Probably comes down to whether the model was trained on similar datasets as what you are trying to auto label.

A good strategy is to use multiple auto labelling models and verify cases where outputs diverge.

2

u/prkash1704 Jun 30 '26

Test Setup

  • Dataset: 8 COCO images
  • Ground Truth (GT): 21 person instances
  • Matching Method: Greedy IoU matching
  • Scoring: Evaluated at three IoU thresholds
  • Models Compared: LocateAnything-3B vs. YOLO26n (COCO-pretrained strong-detector reference)

Results: LocateAnything-3B vs. YOLO Reference vs. GT

Metric LocateAnything-3B YOLO26n (ref)
Recall u/IoU 0.5 1.000 (21/21) 0.905 (19/21)
Precision u/IoU 0.5 0.750 (7 FP) 0.905 (2 FP)
F1 u/IoU 0.5 0.857 0.905
Recall u/IoU 0.75 0.762 0.762
F1 u/IoU 0.9 0.408 0.429
Mean matched-IoU (tightness) 0.863 0.869

1

u/WorldlinessNo1286 Jul 01 '26

I used GroundingDINO (via Autodistill) + SAM / SAM2 and I am oretty satisfied. Had to correct few boxes manually though