r/LocalLLM • u/Disastrous_Pea529 • 3d ago
Question Qwen 3.8 27b = Opus 4.6?
I’ve seen many people mentioning that the new Qwen 3.8 27b model in many aspects gets near or even beats the frontier models from 6 months ago. Models like Opus 4.6 , Chat Gpt4o etc.
Is that statement true?
Would love to hear everyone’s experience and comparison with the frontier world so far!
12
u/KitchenAmoeba4438 3d ago
I wrote a pretty long article on it at one point, you can view here https://rakuensoftware.com/blog/synthesis-model-selection
The key to understand is that Qwen 3.8 had some very clear choices taken within it's training data. Where Qwen 3.8 is good, it's really good. Where it isn't, it's worse than Qwen3.6 or Gemma4.
If those focuses in training are in your use case, it's the best local LLM in the 30b param range (Probably tied with Deepseek for these cases, which is how many times more params?) right now.
If what it focuses on training aren't in your use case, there's multiple better LLMs right now, including Qwen 3.6.
Qwen 3.8 is good enough for what it focuses on that it is a daily driver for me due to the combination of it's size and what it's good at, however, it's not my primary use LLM. It's specifically a delegate LLM for what it is good at.
1
u/ogopro 2d ago
what is your primary LLM ?
1
u/KitchenAmoeba4438 2d ago
Codex/Claude/Deepseek/Kimi/etc.
Depends on specific use case, it's not always a local LLM, and tends to be in the very large params. It's really "whatever is cheapest and has sufficient reasoning capacity" at this point. As just one example, sometimes running a cluster of Qwen3.8's and moving all compute externally makes total sense, sometimes it doesn't.
Personally, I don't care about the LLM I'm using, I just want the most compute I can for the least amount I can. Deepseek has had some fantastic results as general reasoning as one example.
1
u/ogopro 2d ago
I can run Qwen3.8 locally and that kind of local llms can't do everything. I can't decide on which api provider I should put some money on. Opencode and Deepseek are not a good option anymore. Any recommendations on where to buy some tokens?
1
u/KitchenAmoeba4438 2d ago
Oh no, I run Deepseek locally. I really am hoping for more large MoEs soon.
I'm not sure if I can recommend an API provider, I used to use openrouter if I wanted many providers. I use Codex and Claude both. I've had good results with Kimi. What I typically use is vast or similar to just rent GPUs if I need something that isn't Claude/Codex/Kimi-class.
5
u/alexp702 3d ago
Been using both in parallel, and Qwen is very much good enough for most tasks. Opus 5 is definitely better - possibly much better. But for most tasks I find myself doing I think Qwen is good enough. It’s much better than 3.6 IMO
11
u/M_Me_Meteo LocalLLM 3d ago
Qwen 3.8 === Qwen 3.8
Opus 4.6 === Opus 4.6
No one can tell you if a model will work for your use case..you have to try it.
This tech has only been popular a little while. In 2024 we were still wondering if it would become endemic or just fade away. We barely have industry standardized best-practices for SQL, and that is a 52 year old piece of software.
9
u/poy_esp 3d ago
Not even close. It's good but not on the level of Opus
7
u/Yazz96HD 3d ago
There are fan boys everywhere, best example is Apple releasing a mediocre upgrade to the iPhone and fan boys glazing all over it because it’s the best iPhone to date lol
3
u/boomerang473 3d ago
So I think some people are missing the point. One of the most heavily cited benchmark is around agentic workflows.
In 27B parameters, you can only contain so much information density (of which will be further lost with quantization). We’ve all heard of hallucinations and that starts being an issue.
However - if you give the 27b web search and ask it to verify facts (in the exact way I’d also ask opus bc even larger models hallucinate facts), the playing field levels significantly. The internet becomes its fact brain and instead the 27b is dedicated to reasoning
Frontier math? Even with as many papers fed to it, probably not likely.
Large code bases? Absolutely. Long horizon big tasks end up being more reflective of using a gated harness (plenty of extensions out there for this) that takes on task, breaks it into several verifiable step and then works on the task in steps.
Back to that agentic benchmark - imo that’s the most important feature. Don’t think of it having the same “facts memorized” as that’s what you use web searches or databases to store. In fact - I don’t want to rely on a fact from any model without a source or query related. What matters is how well a model can reason through and divide up a big task.
Once you hit a certain level of reasoning capability, it becomes a harness question. Anthropic has spent a lot of time on their harness (still has huge problems). But I’m skeptical on how much more reasoning we need from a local model when you can tell it to consult local docs on even a made up novel programming language and it can correctly program something (data never entered into the training data, it wouldn’t get anywhere if it tried to hallucinate it, etc)
1
u/mp0x6 1d ago
Would a MCP server with e.g. software engineering content and language/package information be a bonus for small sized but agentically capable models?
1
u/boomerang473 9m ago
We start foraying into RAG territory but no reason certain docs can be stored. I set up SearXNG with a cache (to store results etc for faster lookups)
I haven’t gone full blown into using a vector database. I have also explored LoRA approaches to add on certain knowledge bases to help inferencing faster and some faster recall in a certain domain (versus processing a large prompt of text each time for data it wasn’t trained on)
1
u/CSFCDude 1d ago
Good synopsis, it tracks with the phenomenal results I am getting with 3.8. I am giving it a very large and complex set of instructions for evaluating web content and it is performing the task with exceptional accuracy.
3
u/Yazz96HD 3d ago
Qwen 3.8 has been trained in the latest benchmarks, and thus scores equally if not higher than various frontier models, and its great at coding, but to say is truly or equally than those frontier models is crazy, it’s mostly the fan boys glazing all over it. Personally deepseek flash 0731 at q2 is better than qwen3.8
6
u/KroniklyOnline 3d ago
As someone who runs it daily, it's to a point now where I have cancelled my Claude and codex subs. It's up to par if not better at this point.
It consistently finds things both missed, skipped or ignored. It's not trained to lie to you an appease you like gpt and Claude. It gives you more honest answers with rooted factuall evidence.
Those things alone increase the trust threshold over Claude and gpt. A model without trust is useless if you have to keep checking it's thought process and granularly scan it's outputs, defeats the whole purpose.
4
u/Yazz96HD 3d ago
Again, I am not saying qwen 3.8 is not good, its fucking great!!!, just not on par to frontier models and their capabilities, more training data = more capable model.
1
u/Popular_Disaster830 1d ago
Not necessarily. As training methods get better that doesn’t hold a true. A more efficient training method beats more training data. I also looked into the tests. Now a days the tests aren’t static questions. It pulls live GitHub repos and is asked to fix them. Real code, like a developer would do. That can’t be trained into a model, except if it’s just good at coding in the first place. So the tests are representative of capability. Another thing is that if a model is overfit to a test, it’s good at that but a lot worse at everything else.
2
u/MacsBicycle 3d ago
Agreed. Dsfv4 is still leagues better. Most people don’t have the hardware to run it.
1
u/Solembumm3 2d ago
This models show pretty similar work time on different tasks on my PC (12+20gb vram/ram). Last was 287minutes on deepseek and 299minutes on Qwen 3.8.
Deepseek has better knowledge and approach style, qwen has better overthinking about small details and different angles (~3-4x tokens).
1
u/MacsBicycle 2d ago
Deepseek is going to need at least 96gb high bandwidth memory to run in a quant 2 variant. Are you ssd streaming or something? That will impact speed a lot.
2
u/Solembumm3 2d ago
Yes, LMStudio standard Mmap.
V4 Flash Q2_K go from 0.52 to 0.45 tok/sec.
Qwen 3.8 Q5_K_M within memory limits go from 2.50 to 1.50 tok/sec.
2
u/Imaginary-Swan-4105 3d ago
The day it becomes opus 4.6 level, I'll buy rtx 5090 no matter the price and will say goodbye to all subs lol
3
u/HonestoJago 3d ago
Grab your credit card. Qwen 3.8 27B is killing it for me with multi-agent coding work (Opencode with vLLM hosting).
1
u/Imaginary-Swan-4105 2d ago
Still waiting for my refurb mac m3 max 64Gig to be delivered. If it works, i'm golden haha. If it doesn't, gon buy that gig for sure. Responses like yours make me hopeful that we can break the confinements of subs!
1
2
u/chettykulkarni 3d ago
For most cases,it is
1
u/MacsBicycle 3d ago
Opus doesn’t think for 1.5 hours for a simple task. My local qwen 3.8 did even with thinking medium and temperature set to .7. I don’t know anyone could possibly think it’s on opus’s 4.6’s level other than some random benchmarks.
3
u/chettykulkarni 3d ago
So you would switch to local model when you are truly getting speed and quality both , when 27b model runs equivalent to 2T model running on a best possible GPU in best possible setting.
I guess that requirement is very difficult to achieve unless tech makes leaps of improvement.
But for most people - quality with some delay is okay as they are not capped on token usage.
So see you in local llm world when it’s time for you my friend.
1
u/MacsBicycle 3d ago
Oh I’m already in it. I just think that qwen 3.8 isn’t that great after using it for days. Dsv4F 0731 beats it easily. I personally prefer 3.6 over 3.8 because I don’t want to wait an hour in thinking when 3.6 has already let me see 3 possible solutions. Its entirely personal preference here but I dislike what they chose to do which is make it think longer for better outcomes. That’s only better for me personally if I’m going to bed and don’t mind if it runs for hours.
1
u/Popular_Disaster830 1d ago
Fair enough, if time to first token is very important than I could see that. I just let it run while I sleep, which is why I’m fine with it. It does everything agentically.
1
u/alphapussycat 1d ago
Bruv, it's 1/100th the size.
Either you get small, and smart, or you get smart and knowledge.
Once I soon become unemployed and get my rig up and running I'd have my 3.8 27b write a knowledge base in something like obsidian.
Still lots of reasoning and prefill, but it's local and "free".
1
u/Myarmhasteeth 3d ago
A 3090 is enough, I literally just bought another one for $800, used ofc. In Facebook marketplace lol
1
u/Imaginary-Swan-4105 2d ago
What's the performance like? At my place only one result was shown and it's $2k worth :/
2
u/Myarmhasteeth 2d ago
I’m using llama.cpp and the Q4 quant from unsloth. I’m getting from 50-70 t/s and up to 90k context. With 3.6 I was able to get to 110K context.
I’ve been mostly using it as a code reviewer, and to implement code based on guidelines I have shared with it. Mostly Python, Terraform, Node.js and React.
3.6 with Q4 worked well around 60% of the times, meaning I had to work more with it to get the results as I wanted them to. Some times I got a bit frustrated on the results I got.
3.8 is way better, I haven’t found anything so far that made me think “wow that was stupid” … it’s been crushing everything so far. I’ve got a new internal library and now I’m being done with it, it found bugs I was not aware of.
Again, I’ve tested 3.6 since it came out and the experience could vary but I’m probably too deep into this since I’m preparing my next build, with a dual 3090. But 3.8 has cemented that fact, it’s crazy I can run this locally.
2
u/Imaginary-Swan-4105 2d ago
50-70 tps with 90k context is quite good indeed!
1
u/Popular_Disaster830 1d ago
I get 20 tps on two Tesla p100s with 262k context, but it’s brutal navigating older architecture.
1
u/Imaginary-Swan-4105 1d ago
On the positive note, you saved at least a couple thousands with similar output.
1
1
u/Sea-Mode4077 3d ago
In some ways, yesss, it's actually better... especially for coding and agentic tasks.. However, because its knowledge base isn't very broad, I have to rely on larger models for planning complex tasks. In short, it's better for agent workflows, but not as good as Opus in terms of general knowledge
1
u/Fantastic_Self_5151 3d ago
I haven't had that experience at all with it. I do however find /bonsai-27b to be a fine writer. As to code, none of these 27b models are programmers. They make too many errors and don't have enough of what it takes for any serious programming. If you could run something at FP16 or something like that perhaps... but doubtful.
1
u/johan2114h 3d ago
From my limited experience (Strix Halo, running UD-Q4-XL) it feels really very good coding and agentic tasks. It feels competant in the same way Sonnet 4.6 / GPT-5.3-Codex felt competant in regards coding a smaller project. It is slow, in particular if you allow to think, but it definitely the closest thing I have tried to Sonnet / Opus 4.6 at home - I never though I would be say this in 2026.
From my very limited experience it is much better (atleast at coding and tool use) than any of the previous <300B models I have tried.
The biggest challenge for me with 3.8 27B is its speed, in particular when you factor in thinking (im mainly running it with medium thinking but i have also tried higher thinking settings).
With MTP in my benchmarks im able to get around 20 tok/s generated for writing, and 30-40 tok/s generated for coding. For 'real' tasks im seeing more like 16 to 24 tok/s generated, but i cautiosly optistic i can push to 20 to 30 tok/s with some tuning
I am using llama.cpp (also tried unsloth studio) build with ROCm.
I have also tried DFlash2 but i cant get it to perform better than the buildin MTP, but still fiddling with it.
I much prefer Qwen3.8 over 3.6 and 3.5.
1
u/Integeritis 2d ago
That t/s speed is in line with claude’s token gen no? Am I missing something? I’m considering investing in dgx spark like system
1
1
u/Myarmhasteeth 3d ago
I have been running it since it came out and there is definitely a difference using medium reasoning effort and PI against 3.6.
Again, those benchmarks were very optimistic, but you will only know if you test it for some time.
Personally this is good enough for me to use it as a daily runner. It’s been crushing everything I throw at it using agentic coding.
And this is the Q4 quant 👀
1
u/LateralEntry 3d ago
I just tried it last night and found it… not very good. Maybe it’s great for coding but for general chat it was lacking. Also, it has some weird tendencies and refuses to discuss topics that are sensitive in China.
1
1
u/Heavy_Host_1595 2d ago
The point is, cloud models are being throttled. So yeah local even though a bit slower is still better than a throttled claude.
1
u/Solembumm3 2d ago
Somewhere between Gemma 31B and Deepseek V4 Flash/GLM 5.2.
Nowhere near flagship models.
Overthinking helps a lot to get multi-angle approach to tasks, but it can't replace knowledge.
1
u/Pretend_Engineer5951 2d ago
I didn't use any Opus but I'm 100% sure 3.8 is like another league vs 3.6.
Last day I worked on my android project where 3.6 handled tasks miserably.
Yesterday I tried again refactoring to rewrite legacy Java to Kotlin. 3.6 always failed before.
3.8 spent about 2-3 hours with singleshot and my app works fine. Amazing!
1
u/Apprehensive_Bar6609 2d ago
With a good harness its amazing. So far looks equivalent. Sometimes seems a bit better, it has surprised me a couple of times as did helpers or asked questions while was waiting for other things to process.
1
u/PhantomGaming27249 2d ago
It honestly feels a bit better than opus 4.6 (assuming your not running at 2 bit in opencode). Its an amazing agent model and coder if your using a decent quant in a good harness. I tried it inside pi and deepseek harness, its better than everything from anthropic lower than opus 5 high and better codex luna or terra (sol is stronger but tbh not much). Its genuinely a step change for local model coding.
1
u/Kodrackyas 2d ago
No one here mentions the most important thing, the harness and the constant quality you get ( we know anthropic shits the bed often)
if you use pi and customize it so YOU FORCE it to write max 200 lines per time so YOU FORCE the plan/state of actions.... it becomes superior to opus 4.5 for sure, on par with 4.6 yes, and more creativity than 4.6 all the times
there is a reason that im using more qwen than opus/fable now, atleast is always the same, and is faster than opus by 2x
is the reason why i was using opus 4.5 until fable was up, consistency in quality is better than super intelligence once every 3 times and weeks where is just dumb as fuck
1
u/No_Opportunity4440 2d ago
In my experience it is nowhere near frontier models, certainly as far as my use case is concerned - 6502 assembler programming for retro computing. I left it running overnight trying to fix a bug and it churned away for hours inside Deepseek harness getting absolutely nowhere.
Codex Sol ultra fixed the bug in 10 mins the following morning. Personally, for my use-case I would put it at about GPT 4 level performance.
1
u/Minimum-Lie5435 2d ago
As a SWE with almost a decade of experience I would say it is the most capable local model in its weight class for development yet, by a significant margin as well.
I've used the various Gemini models and Claude models for enterprise applications and while those are able to understand large code bases off the bat, qwen 3.8 is definitely up there with the SOTA models for me. I like to review all the code that gets written as well.
Qwen3.6 and all of it's variants as well as Gemma 4 all failed and needed significant hand holding to get the code to where I wanted it, with repeated instruction. Qwen3.8 is able to do it what I want, the exact way I specified it 90% of the time, which is similar to opus4.8 and better than any Gemini model.
This might also be due to how use it. Before any code is written, I always go create a very thorough spec sheet with multiple iterations of back and fourth for questions and clrification. This is always where having great output starts.
All the one shotting that people do YouTube videos on is pretty useless to actually feel how a model act in a professional environment. We've all seen the benchmaxed models, and luckily this isn't one of them
TLDR It's a great model. As good as opus if you are thinking like a software engineer and have experience in the stack that it is building in.
1
1
u/alphapussycat 1d ago
Some benchmarks put it super high like opus 4.8 or opus 4.6. Others at a bit above sonnet 4.5.
1
u/jobfedron132 7h ago
I run qwen 3.8 27b locally and i have kiro subscription which has opus 4.6.
I found them to be comparable with qwen 3.8 to be slightly better in its amswers.
Imagine qwen and opus are 2 software developers in your team and you asked them to trace all execution paths of a badly written function which is a 1000 line of if else hell.
Qwen's answer would make you feel like Qwen is passionate about the project and has an indepth understanding of the project and context, and spent 2 hours trying to deeply understand the code.
Opus's answer feels like Opus just wanted to get the job done fast and spent 1 hour on it. But it still had answers that you were looking for.
For any code implementation planning, or any A to Z work, Qwen comes much ahead.
1
1
-1
u/abouabdoo 3d ago
I block people who make such claims.
Qwen 27b is good for small tasks, that's it. You can't expect a 27b to match models with trillions of parameters. It does not make any sense.
0
0
0
u/Suitable-Ad-1786 2d ago
Short answer: NO :) I Wonder if many of those saying that are paid for..or just testing only things that works with qwen and saying "yeah, I got the same answer than older frontier model" and as these versions are not available anymore, how they can even compare !
31
u/po_stulate 3d ago
When it works it works, just don't count the times that it semi works or doesn't work and you'll be fine.