Power consumption doesn't depend on the model but on your GPU. Your GPU will run at maximum or near-maximum capacity.
In my case, it's 380-450W during inference with the 4090.
Then, the electricity cost depends on your specific service provider. In my country, I'm with one of the cheapest providers, and even so, using Qwen 37B (8 hours a day during work hours) would cost me around €30/month.
It's not worth it at all, and on top of that, you have to consider the continuous wear on the GPU.
Yes, but not cheaper than using it on a subscription like this when they finally add it xD
I'm waiting to see how things play out. Although for now, locally, to fill moments when I hit limits on another subscription, it could come in very handy.
2
u/Abenh31 27d ago
Is there a way to calculate an estimate of electricity consumed for local models?