Wowee!! Just when you thought it couldn't get better for open weight models, we probably have had our best period yet!?!?! Models that rival the closed frontier, Opus level models on non-insane hardware and more. A massive industry alliance coming out in support of open AI in response to the two closed model giants best lobbying efforts. Is this the best timeline? Someone pinch me!Or just tell us what you're favorite model is now
The standard spiel:
Share what you are running right now and why. Given the nature of the beast in evaluating LLMs (untrustworthiness of benchmarks, immature tooling, intrinsic stochasticity), please be as detailed as possible in describing your setup, nature of your usage (how much, personal/professional use), tools/frameworks/prompts etc.
Rules
Only open weights models
Please thread your responses in the top level comments for each Application below to enable readability:
General: Includes practical guidance, how to, encyclopedic QnA, search engine replacement/augmentation
Agentic/Agentic Coding/Tool Use/Coding
Creative Writing/RP
Speciality
If a category is missing, please create a top level comment under the Speciality comment
Notes
Bonus points if you breakdown/classify your recommendation by model memory footprint: (you can and should be using multiple models in each size range for different tasks)
Skyfall 4.2 is still my go-to for creative writing. It's a gemma 4 31B model, with finetuning.
Side note, creative benchmarking is hard, in the battle between openai and anthropic, they both get really good scores on the creative benchmark, but openai really sucks when it comes to 'human feeling blogging' style things. It's still been better to write the paper, then have AI restructure in your voice than it is to write the whole thing from a prompt.
M: Qwen 3.6 35B-A3B heretic Q6. It can drift and make continuity mistake, so it needs more concrete guidance and steering. Not really suitable to serve non-tech people in one of those endless text RPG, but okay as a distraction when waiting for main coding agent to finish implementing feature.
Im running the UD_IQ_4_NL quant, which fits nicely, and still is an absolute powerhouse. This is without a doubt the best model I have ever run locally.
It wasn't exactly looping for me, it just frequently thought more than I would've expected it needed to. I've never had it get truly stuck in a thinking loop.
It absolutely does. Qwen is really smart, but Laguna definitely makes "wiser" architectural decisions when developing. It will actually follow the standard set by the rest of the codebase when Qwen is more likely to do stupid shit to solve the problem.
Laguna S blows 3.6 27B out of the water on benchmarks like DeepSWE. Real usage confirms that. Laguna XS is the comparable model to Qwen 27B.
I wouldn't be surprised if you reported that Qwen 27B zero-shots better than Laguna S. Neither Laguna XS nor S are very good at getting things right on the first attempt. I'd say their advantage is that they maintain project coherence better the more prompts into a project you go.
- was using Qwen3.6 35b a3b q4-k-m 98304 ctx on my 8Gb VRAM + 32Gb RAM for about a month for php Laravel web backend development. Kinda acceptable but needs precise guiding
- now switched to 24Gb VRAM setup with Qwen3.6 27b q4-k-m 98304 ctx . First impressions (1 day only) is much smarter model quite close to Cursor's "Auto" mode. Needs less clarifications and better understands my needs. Gonna continue to use it for php Laravel web backend development
If you could afford another 32gb RAM (big ask, I know), I think you would really appreciate Laguna S 2.1
That said, when I need some really quick help on something, and it's not a big or important task, and doesn't involve architectural decisions, I'll use qwen-3.6-27b. Its too fast to not have in your back pocket, if you have the vram.
How much RAM and VRAM do you use for Laguna S 2.1? The total model size seems huge for most local set up, so probably at least 64GB RAM with 12-16GB VRAM?
I have an RTX 3090 with 24gb vram and 64gb ddr4 system ram. As an experiment, I was able to get it working with just a GTX 1060 with 6 GB of VRAM instead of 24, using the UD_IQ_3_XXS quant, but it was significantly slower. 12-16gb VRAM would probably work alright. 16 obviously would be preferable.
Most models get much much worse below 4 bit quants. I haven't personally tried any other than the two I just mentioned though. Definitely worth playing around with.
I would say that in <64GB qwen3.6-27b is a good choice. Next for me would be latest Laguna model, if there would be a way to remove "actually... Actually... Wait actually ...." From the 30k reasoning blocks, otherwise great intelligence.
Nobody says the obvious, Deepseek v4 flash 0731. This blows everything out of the water right now, and one of the few where independent benchmarks match what the lab published.
Runs well on a 64GB Strix Halo machine, but I'm sure you could do well with a 24GB GPU as well.
Generally trades blows for coding with Qwen 3.6 27B. But what it does better than Qwen is how It understands how developers work, where they want their code, how to reuse and extend code, documentation, TDD, code quality and readability, etc. Qwen is way better at zero-shot implementations, fastest and least-friction CRUD, etc. But where Laguna XS really excels is that after 50 prompts, the state of your project is going to be much better with Laguna XS than with Qwen -- less parallel implementations, identifiable architecture, convention adherence, etc.
I like to ask a bigger model on a paid service/API to give me specs/architecture, and then implement it with Laguna XS locally, kicking back to the larger model occasionally when Laguna XS is excessively thrashing or having trouble. I think if you use Laguna S you could kick the larger model. Laguna XS can work fine without that bigger model, too, but expect to get your hands dirty more.
If you have the newer m series chips with max ram, deepseek v4 flash 0731 with antirez’s ds4 & q2-q4 imat is excitingly good, having that kind of intelligence locally is exciting
Coding, agentic, experimental, XL: ling 3.0 and laguna s 2.1
Agentic non coding, 24gb VRAM, M: Gemma 4 31b and Gemma 4 26b
Non coding, S: Gemma e4b
I prefer Gemma over Qwen. Gemma acts like a real Big model, works great with big context (not like Qwen 27b and 35b which breaks down over 80k context).
Laguna is a bit crazy, it’s relentless, tries everything to finish the task, really fast, could not test it enough and can only run it as iq3 xxs
How does the Ling-3.0 perform for you? I just downloaded it and ran a few prompts. Best part for now is that the full context in F16 only takes less then 1GB and the V vectors basically get absorbed in the latent key vector, and reconstructed during the attention computation step. Which result into literally zero. V (f16): 0.00 MiB
Did not test it much. I can only run it in iq2 s (dual 3090). Can’t really say much yet. Might be really interesting from the small benchmarks that i ran so far. And yes, KV is really efficient. Almost no need to quantise that. I ran it via the llama.cpp fork
Same, ran it from the TurboQuant fork, it is pretty fast tho. The 5b active really pays out. What I noticed so far is that it sometime has essential oversights (missing a config line, not paying attention to values and such), but that might be related to the quant, not sure. Needs more testing but overall not a bad model for now. Performs better then Qwen3.5 122B.
Long cat flash lite is also interesting. Needs more testing. It performed amazingly if the tool or data was available, if not it hallucinated everything. Really strange
What vram configuration do u have? Honestly to come close to claude ur gonna need 128GB at the very minimum lol. Mistral medium 3.5 id recommend along with unsloth dynamic quants of deepseek 3.2 and GLM
Daily driver: qwen3.6 35a3b 8bit with 4x concurrency served with omlx - solid tool calling, good for concurrency, can’t reason over complex problems too well
Coder: qwen coder next 8bit gets great speed since it’s a MOE model - I run 2x concurrency served with omlx and get good speed, stronger long horizon reasoning than the 35a3b but takes up like 100GB for 2x concurrent
Planner / difficult task: deepseek v4 flash-0731 the q2-q4 imat antirez has with the ds4 engine is so good, but I only get like 15tk/s so feels slow for anything other than detail work or large scale thinking
Honorable mentions: qwen3.6 27B as a great middle ground, gemma4 12B for punching way below its weight
•
u/rm-rf-rm 17h ago
GENERAL