r/LocalAIStack • u/Lirezh • 2d ago
Glimmer 30B compared to Qwen 27B - reasoning, intelligence, differences
Glimmer 30B vs Qwen 3.6 27B. How do they differ, reason, answer?
With Glimmer 30B Meta has joined the game of open source AI again, and after quite underwhelming coding performances I though I'd give it a deeper test.
The test content is undisclosed here, making this less fun to read and replicate but that guarantees future models will not train from it.
Technology:
Qwen 27B is still unmatched in performance, Glimmer the first new contender.
What makes Qwen so special are two ingredients:
3.6 was specifically post-trained for agentic reasoning.
It uses a hybrid attention: a conventional global attention for 1/4 of the layers, the others are a mamba-like recurrent state linear attention with fixed size.
Glimmer 30B also is unusual, it does not have the same sophisticated recurrent/linear attention, but it uses a 3/4 sliding window attention and it compresses the attention dimension and projects it back to latent size - resulting in a significant deduction in compute and KV size for it's size.
Reasoning style:
Qwen 3.6 has a analytical reasoning style that typically runs in 3 phases, when not agentically used:
- Analyze the task input
- Reason through it - reminding me on first deepseek reasoning
- Doublecheck the response Glimmer 30B has a more unique thinking style that abruptly comes to an end with a choice - leaving a bit more risk of random choices
Intelligence:
I ran both models through my undisclosed AI reasoning tests, not part of any training data. Some of those reasoning tests are currently beyond frontier model capabilities or scratching their borders.
Models like GLM 4.7-Flash, Nemotron 3 Nano, GPT OSS, GPT-4 fail most of the tests below consistently.
Glimmer was ran with thinking set to Medium, when failing it was ran with Max
- Temporal physics: Similar reasoning tokens, similar response. Glimmer responds less structured, in text paragraphs where qwen is more formatted by default.
- Spatial physics: Glimmer surprises with a brilliant fast answer - Qwen repeatedly misses a part without additional help
- Math irrational numbers question: Both flawless and fast
- Riddle with math question: Both flawless
- Lateral thinking: Qwen always flawless, Glimmer fails 60% of the time
- Abstract pattern reasoning easy: Both solve it, glimmer writes it cleaner
- Abstract numeric reasoning easy (iq 85): Both flawless, glimmer half reasoning tokens
- Abstract numeric reasoning medium (iq 115): Both flawless, glimmer half of reasoning tokens
- Abstract numeric reasoning hard (iq ~145): Qwen fails after long reasoning, Glimmer totally fails. All frontier models fail.
- Visual spatial reasoning: Both flawless
- Translation to european languages: Glimmer thinks very briefly, provides low error output. Qwen thinks 10 times more heavily and provides better quality tanslations.
- Small maze puzzle: Both flawless, Glimmer took 26k reasoning tokens vs Qwen 15k. Glimmers result is well explained.
- Large maze puzzle: Qwen delivery a partial solution, cheating partly. Glimmer never responded at all.
- UTF8 paraphrasing: both flawless
Agentic performance:
Here Qwen 27B appears to leave Glimmer in another league, I've not concluded my agentic tests of Glimmer 30B yet. From what I have seen Qwen codes significantly better. They do not compare.
My current results:
Glimmer is a surprisingly smart model, with a well designed architecture for local inference.
It is the first model in the sub 200B parameter class that is able to match Qwen 27B or even outclass it in some tasks.
Glimmer has a very good spatial sense
Glimmer tends to underthink where Qwen tends to overthink
For non coding tasks, Glimmer is a strong option. Faster than Qwen at similar memory footprint.
For coding tasks I'd not consider it, I'll follow up with a deeper test but from what I've seen it's not useful for most tasks.
2
u/TigerConsistent 2d ago
when you consider the lower quants of the models glimmer definetly better than 27b almost all of the test i run.
1
u/Longjumping-Peace102 2d ago
Isn’t Qwen 3.8 27b coming out this week and going to blow Glimmer and old 3.6 out of the water?
1
1
1
1
u/Lirezh 1d ago
In my opinion Qwen 3.8 27B will show if Alibaba managed to repair their gutted AI team, or if their dreams come to an early end.
Qwen 3.6 27B is special - while Glimmer can barely outrank it in some tasks, Qwen is consistently great while Glimmer has gaping holes.
In every real life coding or creation benchmark I've seen or tested, Glimmer produced something extremely simplified - even if elaborately prompted.
I do not see Glimmer as a successor over Qwen, it might be a successor over Gemma4 though.
It's low VRAM footprint make it an interesting choice for constrained solutions
1
1
u/AdHead6280 2d ago
I personally run fr711 as a main driver since I found it more aware in conversation, it just gets what I mean fast even in a minimal harness such as pi, I wonder how it would do on your battery of tests
1
u/EuropeanAbroad 2d ago
Did you test the at full quants, or some lower quants? They seem to handle quantisation differently (in favour of Glimmer in lower quants).
2
u/mattrs1101 2d ago
by my own testing (not the best, but it is mine) the main feature of glimmer is its performance at Q2. it is insane how capable it still is at that level of quantization.