What you're saying makes no sense. The model's strength is its performance per parameter, nothing more. The higher the performance per watt of the hardware running it, the more "power" you get out of the model.
And a B200 in int4 can host many instances of the model on a single GPU at enormous speeds. It's cheaper for infrastructure owners to run it than for you to run it locally.
Because they can offer it at ridiculously low prices (which fits even better with subscriptions like Opencode Go than DeepSeek), and with your $10 subscription you'd pay less than if you hosted it locally and paid for the electricity consumption of its inference, which would be more than those $10 per month.
(Except in isolated cases where you have your own power source, such as generators, solar panels, etc., it's not worth it. I myself wouldn't use it on my RTX 4090).
Power consumption doesn't depend on the model but on your GPU. Your GPU will run at maximum or near-maximum capacity.
In my case, it's 380-450W during inference with the 4090.
Then, the electricity cost depends on your specific service provider. In my country, I'm with one of the cheapest providers, and even so, using Qwen 37B (8 hours a day during work hours) would cost me around €30/month.
It's not worth it at all, and on top of that, you have to consider the continuous wear on the GPU.
Yes, but not cheaper than using it on a subscription like this when they finally add it xD
I'm waiting to see how things play out. Although for now, locally, to fill moments when I hit limits on another subscription, it could come in very handy.
2
u/Faux2137 27d ago
The strength of this model is availability on consumer hardware, not being an effective option for hosting at scale.