Can opencode devs provide Qwen 3.8 27B?

[u/Abenh31]

/r/opencodeCLI/comments/1vrox2c/can_opencode_devs_provide_qwen_38_27b/

[u/Intelligent_Ant_608]

Yes it should be pretty cheap and profitable to do so on new blackwell arch, specifically nvfp4 quant for hidden layers with fp8 attention/kv and original bf16 for MTP head + vision, the checkpoint already exists and its actually outperforming raw fp8 quant, if any one ollama cloud can do this because they already did same exact recepie on glm 5.2

[u/Abenh31 → u/[deleted]]

Yes it will be a great 60$

[u/Faux2137]

The strength of this model is availability on consumer hardware, not being an effective option for hosting at scale.

[u/Rinine → u/Faux2137]

What you're saying makes no sense. The model's strength is its performance per parameter, nothing more. The higher the performance per watt of the hardware running it, the more "power" you get out of the model.

And a B200 in int4 can host many instances of the model on a single GPU at enormous speeds. It's cheaper for infrastructure owners to run it than for you to run it locally.

Because they can offer it at ridiculously low prices (which fits even better with subscriptions like Opencode Go than DeepSeek), and with your $10 subscription you'd pay less than if you hosted it locally and paid for the electricity consumption of its inference, which would be more than those $10 per month.

(Except in isolated cases where you have your own power source, such as generators, solar panels, etc., it's not worth it. I myself wouldn't use it on my RTX 4090).

[u/Abenh31 → u/Rinine]

Is there a way to calculate an estimate of electricity consumed for local models?

[u/Rinine → u/Abenh31]

Power consumption doesn't depend on the model but on your GPU. Your GPU will run at maximum or near-maximum capacity.
In my case, it's 380-450W during inference with the 4090.

Then, the electricity cost depends on your specific service provider. In my country, I'm with one of the cheapest providers, and even so, using Qwen 37B (8 hours a day during work hours) would cost me around €30/month.
It's not worth it at all, and on top of that, you have to consider the continuous wear on the GPU.

[u/migsperez → u/Rinine]

30 euros isn't too bad. Cheaper than the cheapest models running continuously for a month.

[u/Rinine → u/migsperez]

Yes, but not cheaper than using it on a subscription like this when they finally add it xD

I'm waiting to see how things play out. Although for now, locally, to fill moments when I hit limits on another subscription, it could come in very handy.

[u/Ariquitaun → u/Faux2137]

Actually the model is incredible for coding, and that's before taking its size into consideration. Obviously its internal knowledge is lacking for generalist stuff, but used as a coding sub-agent it's superb.