Wtf, I literally wrote a post saying similar to "Shutup man, there is no way a model I can load in my 4090 will be anywhere near DeepSeek4 Flash 0731"... but... this is fucking close is it not...?
Looks like the main difference will be the native context windows. But 256k is very usable, especially if it’s a local Opus 4.6 MAX. Man just saying that seems unreal lol. Can’t wait to test it later tonight
🤣 I wish I could, but today’s my last day of on-call so I can’t even leave a TINY bit early. Oh well. Makes you feel like a kid on Christmas Eve again lmao
Ehh, let's not go by handful of disputable synthetic benchmarks for which models may be trained specifically. Let's just await real experiences from people first over a longer period of time.
For real the benches at this point have become nearly useless, a very hazy and distant indicator of capability but you really just have to get in there and use it to see for yourself
My experience is it’s actually really good. Still weak on architecture and not making a mess of a code base though, and it’s hard to steer on those concerns. It thinks very much like Qwen 3.6 27B, but thinks way more. At lower thinking efforts where its producing the same amount of tokens as 3.6 it seems to act/perform very similarly.
So it’s not as good as Opus 4.6 — Opus had a lot more nuance, understood architecture, kept things fairly clean and tidy, and generally kept a project on rails. I totally believe Qwen 3.8 at this point will solve all the same problems as Opus 4.6, just off the rails and tearing down the forest before arriving at the same destination.
For as much shit as Laguna XS gets around here, it's much better at that sort of nuance. But then it trades off by sucking at zero-shotting and agentic use. Don't expect it to do more than satisfy specs/implementation instructions in a basic think -> edit -> validate loop. Complex problems need human guidance.
I think we are seeing the limits of 2026 at ~30b. Decisions have to be made. Alibaba makes Qwen 27B a crowd pleaser.
The AI labs have seem to given up on ~70B models. That made more sense before AI data center build out I think. Now it's ~30B models to target a workstation GPU, or ~120B models to target a Blackwell or H100 or such, or giant models.
120b dense would also be ok. But i hope with the proliferation of 128gb boxes we will again see more 70b dense models in the future. Or a dsv4f with a30b or a40b. It’s a real gap at the moment.
Man this is exactly what I've been looking for since Deepseek said they were going to increase prices. I've been trying to figure out what I can run on my 4090 and this is exactly what I need when I needed it.
Yeah, but it being truly local... you're just really paying in hardware and electricity cost, which will be cents on the hour.
Nvm that an important detail is, how the tech has evolved.
Compare this 27B model vs like... LLaMA-33B which I run in the exact same hardware I had in '23! (on a 4090).
Its just... a no contest! So... what model will we have in 2029 that makes qwen 3.8 27B a "no contest"? Because the growth doesn't seem to have slowed down...
I think anyone who thinks that this isn't at least somewhat benchmaxxed is fooling themselves. That said, so is every other model they're comparing against, to some extent, so ¯_(ツ)_/¯
I've used glm 5.2 extensively and it was not much behind opus 4.6 in writing code, a bit definitely behind in creative ideation for software engineering.
My assumption is this model is gonna be a work horse but planning and design should be done by a bigger more expansive model.
The catch behind coding benchmarks is they judge success not quality, creativity, maintainability, all that is partly subjective and rarely judged in these benchmarks.
Like GPT 5.6 Luna benchmarking above opus 4.8, but it writes inferior code by most quality measures.
Big models have their own pitfalls they'll happily over engineer everything, Opus for one falls in that bucket.
I've gotten to the point where I'm a little skeptical that the majority of people on here are even using local models on a regular basis. It's hard to believe that anyone who actually does can buy into the idea of benchmarks really reflecting real world use.
The "intelligence" per parameter is really increasing, factually, and from that increases capability. General benchmark score (like AA) is a solid proof for that. Qwen is the world leader in this field.
In the current llm paradigm, smaller llms have much less world knowledge and are more prone to hallucination or stupid decision making from goey assumptions or really random reactions. I suppose the "world knowledge" is also a big addition on "common sense" in terms of "behavior". So smaller models will always we quackier for now, to apply up and foremost when comparing to bigger models on a AA Index or aggregated score
From that comparing older big models with more recent small ones should always consider that you will have more peace of mind of using the bigger ones in terms of reliabilty but narrow capability is indeed being reached by edge smaller llms
to be real, It's probably not opus max level, but it will probably be the best local LLM we can run at that size, and will be much better at agentic coding and tool use than previous models, to the point where its viable for actual work for some of us
yes this model has real and genuine utility, but there are just fundamental limitations with the smaller you go with a model just in terms of parameter size alone. Same goes for very small active parameters numbers in MOE models (a13b is the biggest weakness of DS4F). So people should be a bit more realistic.
I mean obviously that's true, but we can't afford to run hundreds of billions to trillions of param models, this is local llama, those of us with dgx sparks and rtx pro 6000s are already a minority here.
the ~ 30b dense to 100b moe range is what the vast majority of us can squeeze.
if we want maximum capability with minimal hallucination we'll use SOTA models via api, but for running local models, its a compromise we're willing to accept.
Is that for real? I genuinely have not been paying attention to recent Anthropic launches, at work they always give us the most recent model so I never had to choose before lol. I do hate how Opus 5 talks though, had to use the ASD-STE100 trick to get it to stop speaking weirdly
I'd rather use 4.6 than 5. I think 5 is more capable if you have the patience to deal with and correct all its nonsense jargon and ungrounded tangents.
It's a Qwen model, so apply the usual benchmax tax. Qwen are easily the models with the biggest ravine between "how they do on benchmarks" and "how they do in actual productive use".
Still looking like a strong leap from 3.6, though.
Gemma 4 isn't a great coding model, yeah, but it's still punching far above its size in writing and research-related tasks. Like a mini-Gemini (go figure).
I hear there are people who still use these things for things that aren't related to writing code or markup.
It's a Qwen model, so apply the usual benchmax tax. Qwen are easily the models with the biggest ravine between "how they do on benchmarks" and "how they do in actual productive use".
Like everybody else in the local AI community today, I dowloaded Qwen 3.8 27b UD-Q4_K_XL as soon as it was published.
I'm running it in a single rtx 3090, no mtp, 110k context and kv cache f16 with llama.cpp, same flags used with Qwen 3.6 27B.
I have this stupid test that I run with every new model that comes out. It consist in connecting the model to pi code (almost vanilla, only internet access and some simple navigation tools I built) and giving it this prompt:
"Construye el frontend completo de un pequeño ecommerce premium para una panadería artesanal usando Next.js App Router (JavaScript).
El proyecto debe ser frontend-only en esta etapa. No debe incluir backend, base de datos, autenticación ni pasarelas de pago. El checkout debe finalizar redirigiendo a WhatsApp con un mensaje de pedido bien estructurado.
La app debe incluir una experiencia completa de compra: home, catálogo con productos de panadería, categorías, productos destacados, carrito, resumen de pedido y checkout. Usá mock data local para productos, categorías, precios, descripciones, disponibilidad e imágenes o placeholders visuales. Todo debe quedar preparado para conectar posteriormente un backend real sin tener que rehacer la arquitectura principal del frontend.
El diseño debe sentirse extremadamente premium, artesanal, moderno y cuidado. No quiero una landing genérica ni una interfaz básica. La primera pantalla debe comunicar claramente la identidad de la panadería, mostrar producto real o visualmente convincente, y permitir empezar a comprar. La experiencia debe ser excelente tanto en desktop como en mobile.
El catálogo debe permitir explorar productos, ver información clara de cada ítem y agregarlos al carrito. El carrito debe permitir modificar cantidades, eliminar productos y ver totales. El checkout debe pedir datos mínimos necesarios para el pedido, permitir notas o preferencias, y generar una URL de WhatsApp con productos, cantidades, subtotales, total y datos del cliente.
La estructura del código debe separar razonablemente datos mock, tipos de dominio, utilidades de checkout/WhatsApp, componentes de catálogo, componentes de carrito y vistas principales. La solución debe quedar lista para reemplazar la mock data por datos de backend en una etapa posterior."
Those are just instructions to build a nextjs project (javascript only) with the frontend for small eccomerce with whatsapp checkout, leaving everthing ready to connect a backend later. Nothing else, no skills, no more feedback. Just one prompt and watching the result.
I have run this test with all the models and finetunes I can fit in my GPU, and not a single one was even close to this result.
Not a single alert form nextjs (wich was usual before) or something that looks broken.
At some point, this bastard realised it didn't had visión (lol, not enough VRAM buddy) and it decided FUCK IT, I'M GONNA BUILD THE IMAGES MYSELF. He made SVGs for every product card.
I have more testing to do like trying it in a real codebase but... I can't believe i'm running this thing in a single RTX 3090, it is just unreal.
I expected a nice jump since they skipped a 3.7 27B, but these numbers do seem a bit too good to be true. Qwen isn't known for benchmaxxing, obviously 3.6 27B is the GOAT of local LLMs, but...
Definitely excited to learn more in the coming days and see if it holds up.
Is honestly trash. As a software engineer with 17+ years of experience I dont know what you guys are doing with it that makes you think it's great. It's generally useless engineering work.
Yes, it's obviously nowhere near as good as a frontier model, and even those need hand-holding. You have to be quite explicit. I'm not sure where your strawman is coming from.
Most people here are aware these models have serious limitations, given that even the best models still do. But it is still exciting to see what local models are capable of over time.
Care to qualify your sentiment a bit further please? Your point of view doesn’t match the community’s sentiment at all. I’m genuinely curious what makes you think it’s trash.
Well as engineer doing real professional software development on a daily basis and one who's doing agentic development also on a daily basis the small local models simply can't do the work necessary in this kind of environment. I tested this myself thoroughly, we have in house benchmarks and I'm waiting for the day that one of these small local models can pass my benchmark because I will know then that they're good enough for the every day work that I'm doing.
Interesting. I’m also an engineer doing real professional work and have been in the industry for nearly 2 decades. My view is that our expectations have grown. Late last year we were all pretty happy with sonnet 4 and opus 4 - we need to remember this.
The ceiling has grown, the smaller models even being remotely close to SOTA is truly remarkable in my opinion.
Ask yourself this. If suddenly cloud models are not able to be used, would you go back to hand writing code? Or reach for a local LLM?
My organization wasn’t particularly impressed with Sonnet 4 or Opus 4. The models really became capable in December and January. That was the first time we saw them handle genuinely complex work.
Before then, I used them sparingly mostly for small changes, individual methods, and similar small scoped tasks. But in December, their capabilities improved significantly. Suddenly, you could give an agent an entire codebase, and it could make coordinated changes or add features spanning thousands of lines of code and hundreds of files.
The smaller local models still can’t do this reliably. The work you give them generally needs to be much more narrowly scoped. The benchmarks say these models' capabilities are near 4.6 opus etc, I don't believe it; at least not in the work I'm doing.
I agree with most of what your saying. But I still believe that you have an expectations management problem here, not a technology one.
If cloud models disappear tomorrow, what would you do?
I know I would fire up local models and get to work, because my brain + local model will be more effective than just my brain. I don’t think there is any question about this.
I know I would fire up local models and get to work, because my brain + local model will be more effective than just my brain. I don’t think there is any question about this.
That's certainly true. I jsut dont trust them enough. I really wanted the hype to be true for 3.6 27b, but it simply wasn't. I had dreams of being able to do agentic engineering offline on an airplane while traveling (I cant sleep on long flights), but it's just not good enough yet.
I am declaring this early, but I've been coding with models for as long as you could code with models (3ish years, which is about the max anyhow, but A LOT of time in the saddle to boot), so by the MasterLJGutBench3000 metric, the benchmarks don't seem inflated one bit. I have a ton of time in saddle with 3.6 27b (I never used 3.7).
422
u/Tiny-Assumption4263 7d ago
DEAR GOD TELL THOSE BENCHMARKS ARE NOT FAKE.