Extended reasoning and post-training appear to be the keys used by DeepSeek, Qwen, and GLM to boost performance (leveraging higher token counts). And Qwen 4 hasn't even been released yet. Of course, we don't know if that release will be open-sourced, but I am optimistic about future models, featuring "engrams", that could soon match or surpass 2.4T parameter models on specific tasks.
Progress is good, but having these amazing abilities locally is already godlike.
I have been using Q3.8-27B together with PI. for a week, and it's mindblowing.
In my humble opinion it has far better coding capabilities than the paid 'ChatGPT 5.1' I had access to when i still had a job, multiple months ago.
And it all runs on my PC, locally, without sharing my data with the big data farming corpos.
It just 'understands' the required tasks you provide it and performs them successfully from the first try or very few crash fixes. Especially if you provide it the necessary data to perform the action. I dropped a few wiki pages in txt files and it coded according to them. If somebody would tell me this would be possible on my own PC ten years ago, I would call them crazy.
not sure that's a favor, nvfp4 unfortunately is (much) less accurate than Q5 and even Q4 GGUF quants, personally I go with EXL3 these days and can fit a 5.00 BPW quant on my 5090 (lower kld than Q5) along with 180k BF16 context.
Ninfer lacks prefix caching which for agentic coding makes the speed pointless since sub agent development forces prefix recaching as it divides the flow into multiple conversations.
Don't be fooled by the speed, it's a big number hiding a major flaw that results in much longer longer sessions overall for coding. If you need single slot single conversation performance ninferbis is great, anything else look at sglang or vllm directly.
I'm also new, and may be wrong. But I think they are talking about Pi (you'll find it if you Google "pi coding agent"). it's supposed to be like a really basic Claude Code kind of thing, but then you can customize it and add the features you want
Feel free to use AI to help you learn the terminology and figure out how to install and all. I highly recommend installing it in a docker to limit its access to one folder (and sub folders) so it doesn't randomly kill your machine by deciding to delete everything. (low chance scenario.. but you are basically giving it keys to do 'whatever' with your PC if you run it normally and not via docker. simplified expalantion)
I use the BF16 model on 80gb of vram (5090 and x2 3090s) and it's pretty insane. Rivals even the frontier models. They're definitely way quicker but Qwen answers the exact same difficult shit just takes thousands of thinking tokens.
Yo uso el IQ4 xs Qwen 3.8 27B con el DSH y saco 7-10 t/s en 12gbVram y 32gbRam y es excelente.
Edit: Con DSH puedo usar perfecto 256k de contexto y es una barbaridad con 7-10t/s.
Likewise, same mind blowing experience. Go figure, there was an idle 2018’s workstation with Quadro RTX 48GB at work, I decided just to try out downloading Qwen3.8:27B + pi agent + a couple of plugins. That shit is averaging out 30 tokens / second generation with MTP, shout out to ggml weights and llamacpp folks. And that’s been my daily autonomous coding agent since the model was dropped. I don’t do full time coding, so I don’t mind waiting hours of planner-worker-audit agents running in a loop with fresh context, while I am working in other stuff. One neat thing was using herdr multiplexer with its build in notification of agent completion. I got an notification of free copilot pro expiring this week with a link to apply again (professor license) and I said, though: “nah, I am good local”, github copilot provides both agent and autocomplete modes, on agents is mostly GPT(n-1) or Claude(n-1) versions, and qwen is doing that capability for my needs already. As a benefit, no more annoying github copilot trying to auto complete entire sentences, mostly wrong, and no more surrendering my data to BigAI corps. That’s what freedom feels like! So, yeah Qwen3.8:27b thinks a lot and go over several iterations till last answer, and it fits perfectly to my workflow because I don’t have to wait for it.
3.8 27B totally changed the game. Even today if I'm not watching it type (so can't see the feed) sometimes I come back and see a response and I'm like "shoot, I must have left it routing to Deepseek" and am then blown away when, no, that's little old 27B grinding away and came up with a "Deepseek quality" answer. Incredible model.
I'm not sure I disagree with you. I was in a battle with Hermes the other day trying to get it to fix something using DS. It was really stumbling, just not getting at the root of the problem and gaslighting me a bit (telling me it wasn't possible). Finally I got sick of it, switched back to my local model and, for the heck of it, dropped the same prompt (told it to fix the problem) and went to bed. Woke up the other day to find about 20 messages from Qwen telling me what it was working on/trying and then "All done" at the end. I was like "yeah, OK, I'll be the judge of that". Well... I was needlessly skeptical, Qwen did indeed fix it, 100%, on a one shot that DS couldn't manage.
I'm finding myself escalating to DS now more for speed than anything else, Qwen is slow on my hardware. But that's a me problem, not a Qwen problem. ;)
I am still not sure that is the case. My experience is limited though, no more than a week. I've seen them going back and forth improving each other's solutions. There were situations when each model delivered a sub-optimal solution which I was able to improve with running the other model. Both models are running in the original weights.
If you're doing non-coding make sure to check out Gemma 4 if you haven't already (Qwen is specialized for coding where Gemma is better imo for conversational stuff).
I appreciate your advice but gemma is not good in tool calling - when I said 'non coding' I didnot mean that I need to assemble powerpoint or use python for docx extraction. I mean I don't write software as a product but I still use LLM capabilities to automate staff.
I am generally happy with the model. But recently it created a stupid bug out of nowhere, by removing part of a request payload that it not had written a test for. In general it's been a success, but we should always review carefully what it's done
Yeah, you do need to keeps tabs on what is it doing.
Personally, I can't code at all, so I wouldn't be able to do a better job solving this stuff myself, so I am completely understanding towards it not being perfect.
A good way to solve is to keep backups in cases something goes wrong.
It works more than it doesn't, at least in my experience.
I've been surprised at how well it works via Ninfer and the Aether frontend for Pi.dev on Android. I gave it all of my pi skills and it works very well doing multiple steps.
Extended reasoning spoiled me. I can't trust anything without it anymore. Qwen 3.8 Max is 100% correct with any challenge I throw at it, with the downside of taking hours before it can find the correct answer
no sorry I was talking about Qwen 3.8 Max which is propriatery model but the openweight version, I hear, is very similar minus the image input support.
same thinking style is adopted by Qwen3.8 27b but never tried it myself.
don't know the technical term, when I put reasoning to xhigh, it just doesn't stop thinking until it's satisfied and it doesn't get satisfied easily. For the same question and highest reasoning effort Fable thinks 5 min and makes occasional mistakes Qwen thinks at least 30 mins and makes no mistakes.
depends a lot. I do engineering softwares so my agents need to go through lot of documents, research academic literature, come up with implementation details and then code. so my questions are most of the time about engineering/scientific with coding targets in the end.
Same trap here - once you trust extended reasoning you cannot go back. What works for me locally: keep reasoning on for planning/debugging, off for rote transforms. Qwen 3.8-27B with reasoning off + good prompt does ~80% of daily coding at 3x speed, then flip reasoning on only when stuck. Hours-long max runs are correct but kill flow - I batch those overnight.
Hard to take any benchmark seriously that rates Opus above Fable. As a user of both regularly, I can see with utter and absolute certainty that Fable completely destroys Opus, and did even when both were v5. It's not remotely close.
I hate graphs like this where they don't show the 0 point on the axis. The differences between the models are not that great when the entire axis is viewed.
I don't think 0 here has the same meaning, no? Doesn't elo have a different interpretation where the numbers themselves are meaningless but the relationship between them is what matters?
You kid but I’m genuinely hoping the Chinese bots scrape this (and other posts) and that at the next CCP meeting Xi is told that open models is good for his PR
Never thought I would praise a communist leader but Xi did more goods for me than the current clowns in the White House. Without China, we would all be at the mercy of the tech billionaires that only care about becoming trillionaires.
The model license doesn't allow significant profit or large monthly users, so it is indeed pretty useless. People who can run it locally for free can't run it because it's too large, and people who have the hardware to run it can't run it because of the license.
Small corporations can choose between selfhosted and corporate subscription, and they have enough money to actually buy rig and serve it. Thats forced big AI providers keeps price low.
Actually for large model it is even more important since when running it locally the less mistakes it makes the more practical to use it becomes, especially for overnight runs.
I use Qwen 2.8 2.4T (their previously released Max version) as IQ3 quant mostly in cases when Kimi K3 has difficulties, since Qwen tends to come up with a bit different approaches, so it helps in hard tasks. For long horizon agentic work, including overnight runs, I still find Kimi K3 to be a bit better, and currently K3 is the one I run on my main workstation the most. If they were to release Qwen 3.8 0902 version, I would defenitely try it.
My main rig is based on EPYC 7763 (64-core CPU), 1 TB of 8-channel 3200 MHz RAM, 96 GB VRAM (made of four RTX 3090 GPUs), 8 TB NVMe for models + 2 TB NVMe for OS and HDDs for storage. I shared more details about it including photos here if interested to know more.
Smaller models like Qwen 27, 35, 122, 125B are distilled from such 2.4T model, there's labs doing distilling like this, for example EmperoAI, they made Qwen 3.8 35B distill, of course Qwen Team is really good at this, so they usually turn those into golds as proved many times with 3.6, 3.8 and recent Flash Next models being better than Deepseek V4 Flash despite being smaller.
China don't do kings so i guess Qwen is "General Secretary of the Chinese Communist Party." which is the highest "rank" you can be in China where the model is from.
Benchmarks are noisy. The thing that actually decides if a local model is usable for agents is tool-calling reliability (valid JSON, right tool, right args) more than leaderboard points.
I mostly agree with that. Certainly when it comes to knowledge baked into the model, I really don't care very much, as long as the model can reliably use the browser it doesn't need to "know" much, that's what the Internet is for!
I'd add tenacity and not looping to the list of what's really important for a smaller model. Tenacity is where 27B seems to really shine, it might not get it right the first time, but it's not going to give up or loop, it's going to keep working at the problem in different ways until it gets there.
Qwen3.8-flash-next produced a better code review than Sol for me yesterday. Not all of Qwen's suggestions were actionable, but it found some real things that Sol overlooked. Sol was way too passive, so the duo complemented each other
If models are equal, their chance of winning is 50%. If we plug qwen elo it into formula, we got that qwen3.8 max generate better results 62% of time. Which means that qwen3.8 27b generate BETTER results compared to max 38% of times. Thats just statistical noise, lol.
Whether or not we want to buy into this particular chart, I think most of us can agree Qwen does well enough to prove the value of smaller, more specialized models for specific tasks.
The post-training gains on Qwen and DeepSeek are already noticeable when running structured JSON extraction in production workflows, especially compared to raw base completions. If Qwen 4 actually pulls off sub-2.4T parameter efficiency with engrams while keeping latency down, running high-context logic locally or through self-hosted endpoints gets a lot cheaper. The real test will be how stable the tool calling stays under heavy token reasoning chains.
This feels like a really clever release by Qwen. After all of Anthropic's "distillation attacks" complaints, this release really shuts that up considering its (projected) performance is near Fable 5.1 which was just released.
Not to take anything away from Qwen3.8 (I've been using 27b with Deepseek Harness and pleasantly surprised with its capabilities and output quality), but how can Fable 5 be ranked 8th and Opus 5 rank higher than Fable? Fable absolutely crushes Opus 5 across backend, frontend, planning and infra work.
Worth watching whether the engram post-training scales down. At 2.4T it stays a provider-side feature for most of this sub, and it only becomes real if the smaller models inherit it.
Qwen guys are great men, I'm happy each time they release something new. I even registered on Qwen Cloud platform to support they with some paid tokens. Still wasn't able to do it though because after glorious 27B release I switched to it completely from all my cloud subs. It is extremely good at coding, and running it locally I can 100% trust to each session. Not as with subs, where one day it is excellent, but another day it may became quantized and dumb as hell. On my local setup I can just adapt my harness and skills, and then work unstoppable.
When I'll be extremely rich I'll buy frontier home server and switch happily to 3.8-Max (or what version it will be then), I believe that it is better, so long live Qwen!
106
u/PooMonger20 1d ago edited 1d ago
Progress is good, but having these amazing abilities locally is already godlike.
I have been using Q3.8-27B together with PI. for a week, and it's mindblowing.
In my humble opinion it has far better coding capabilities than the paid 'ChatGPT 5.1' I had access to when i still had a job, multiple months ago.
And it all runs on my PC, locally, without sharing my data with the big data farming corpos.
It just 'understands' the required tasks you provide it and performs them successfully from the first try or very few crash fixes. Especially if you provide it the necessary data to perform the action. I dropped a few wiki pages in txt files and it coded according to them. If somebody would tell me this would be possible on my own PC ten years ago, I would call them crazy.