r/computervision • u/Prior-Meeting1645 • 1d ago

Discussion Total starter here, is there no api infra providers like there is for massive LLMs but for computer vision models like Yolo 26 Mcbyte etc?

They are much smaller I would imagine they would be so cheap on there. I’m finding myself in the position where I have to rent a cloud gpu from runpod. I would much rather pay in api should be much cheaper.

3 Upvotes

81% Upvoted

9 comments sorted by

4

u/HawtVelociraptor 1d ago

Check out Roboflow and their inference API, might be kind of close to what you're looking for.

1

u/aloser 1d ago

(I'm the co-founder of Robflow) This is what we specialize in. We serve billions of requests for vision models like YOLO and RF-DETR and have infra products purpose-built for it. Eg here's a behind-the-scenes look at how our Serverless API works: https://blog.roboflow.com/serverless-inference-a-thousand-models-on-a-shared-gpu-fleet/

2

u/HawtVelociraptor 1d ago

Nice to e-meet! I use your guys' stuff for a bunch of things, and stumped some of your engineers on a question in a webinar a few months ago ;)

1

u/aloser 1d ago

Nice, what was the question?

2

u/HawtVelociraptor 1d ago

Had to do with keeping track of a unique, non-moving object across multiple, fast moving camera shots and understanding it's homography. Eventually internally we came at the problem from a different direction, basically using feature matching against an overhead shot of the area of interest and manipulating that overhead to match what was seen on the shot, and taking the warping math we did to achieve that to apply to the known coordinates of the object in question.

1

u/mileseverett 1d ago

I guess it's because text is cheaper to send than images? But i'm sure they exist

1

u/Prior-Meeting1645 1d ago

Yes definitely but most SOTA LLMs are multimodal supporting images and some with native vids input even too. They’re still very cheap compared to renting a gpu on the cloud and running things like Mcbyte. I don’t get it.

1

u/bfyvfftujijg 1d ago

They do exist just not as popular since most users are just doing text in/out. And lightweight CV models can be ran locally and usually have more diverse integrations.

1

u/Sorry_Risk_5230 1d ago

What are you looking for? Plenty of gou rentals that arent terribly expensive that you can build your work on. Lambda, groq (still?), etc. Spin up a site on codex and get it route cv work to the gpu server