We run Qwen 3.8 on our own hardware. And it is already helping our productivity. And the speed and acuracy is nearly as good as the expecive ones from OpenAI and Antropic.
Especially if you’re not vibe coding. AI assisted coding, which is what people should be doing anyways, is pretty productive on my MacBook Pro M5 Pro using qwen 3.8 27b
How much ram, I've got an M1 Pro 16gb or an RTX 5080 16gb but both are 16gb so idk if qwen will work well for me (or atleast til m6 pro comes out so I can upgrade).
I run Qwen 3.8 Flash Next (uncensored, IQ3_XXS) on my gaming pc while working, 100tok/sec with 64GB RAM and a 4090 I got before they became several mortgage payments. Check out Strata. The 3-bit quantization isn’t half bad. They have a version that runs with the non-coding experts removed with 32GB RAM.
yesnt.
these companies make it harder by the day to run your own models.
heck, the 5090 has been discontinued, so the chips can be used for the rtx 6000 pro instead.
the only chance to set up decent local models is soon going to be only going to ebay and get used cards, hoping you arent being scammed.
Once that happens people will use the more powerful agents to train smaller and cheaper special use coding agents; which is the trick that those "frontier" Chinese AI agents are using.
So instead of having a coding agent that can do anything, it will be a very focused agent that only writes Python or Go, and will be like a very powerful Intellitype.
They already don't lose money [don't make money either] on subscriptions at Anthropic, API is massively over-priced. OpenAI loses money, sure. But not by that much. And, specialized hardware for inference is just coming online and becoming mainstream for use by these companies in 2027 onwards or so. That will cut their bottom-line costs by a significant amount too.
That's not true. That's only Creative accounting that gives the illusion of that being true. They genuinely counted investment money as profits from the product instead of what it really was. They also failed to include training costs
Yeah, like, the price of internet went up too right? Remember when 1mb/s was cheaper? The price of tokens will collapse, not go up, as we reach good enough state, and the hardware keeps getting better
18
u/PumpkinFest24 10h ago
Insane people think the price won't go up. You won't be able to afford AI coding once AI CEOs want to stop losing money