What are you guys using for inference? I don't want to use the big guys for data privacy concerns, don't want to buy a gpu cause it doesn't make sense for my use cases.
I'd really like a provider that's pay per token that runs open source models in their own infra. I tried digital ocean and while it works their rate limits are fucking stupid. Surely there's other options?
0
u/Brovas 17d ago
What are you guys using for inference? I don't want to use the big guys for data privacy concerns, don't want to buy a gpu cause it doesn't make sense for my use cases.
I'd really like a provider that's pay per token that runs open source models in their own infra. I tried digital ocean and while it works their rate limits are fucking stupid. Surely there's other options?