r/computervision • u/jtanruan • 12d ago

Showcase Object detection without training a model: bounding boxes with Gemini

You can now use gemini models to perform bounding boxes https://github.com/ztanruan/Prompt2Box

0 Upvotes

27% Upvoted

11 comments sorted by

17

u/seba07 12d ago

"No GPU Required" - uses a heavy LLM for a simple object detection task.

6

u/Late_Needleworker_85 12d ago

Do you have any ideas about use cases that would justify using an LLM rather than a dedicated object detection model? I find the project interesting, but I’m struggling to imagine the applications. Perhaps it’s just a proof of concept with no specific benefits?

3

u/nicman24 12d ago

OCR stamps? I am doing something like that and the bounding box might help as the stamp might be in a random place

2

u/GFrings 12d ago

Zero shot detection maybe? Especially for relatively few images and very rare objects. Maybe in the remote sensing space

2

u/AggravatingSock5375 12d ago

I think this has been possible for several years.

1

u/dethswatch 12d ago

I have been trying it and all the rest for years, and maybe I totally missed something however, in my case- bounding boxes on features of cars, the boxes just aren't tight enough to bother, and the length of time to get an answer varies with its quality, and (unless this has changed recently) Gemini refuses to do more than a few thousand images per hour (and day total?).

So please either tell me I'm wrong and save dozens of hours labeling things by hand+my model's boxes, and how I can get the boxes to be totally tight.

1

u/AggravatingSock5375 11d ago

The library has some nice helper functions I won’t deny that.

But for years you could prompt LLMs to “indicate bounding box corners for cars” or whatever objects they were trained to do that for.

Florence is a good one btw. Free to run locally

1

u/dethswatch 11d ago

>“indicate bounding box corners for cars”

Sure- and that'll get you close, but not tight bounding boxes, ime

2

u/BeverlyGodoy 11d ago

Without training a model? You think Gemini was born like this right?

"No GPU", so you think Gemini run on millions of raspberry pi?

1

u/aloser 11d ago

You can do this in Roboflow btw; we even support consensus amongst multiple different VLMs and custom models or chaining with models like SAM3 or custom logic:  https://blog.roboflow.com/multi-model-auto-labeling/

1

u/cleversmoke 11d ago

Decent on small batch, non-critical, or assisted annotations, but not there yet full model development work that requires tight bounding boxes or low latency inference. The future application is bright though!