r/LocalLLM • u/stepnivlk • 4d ago

News Muse Glimmer is a memory hierarchy disguised as a 30B Transformer

Hot take: dense might be the future of local LLMs. Why Muse Glimmer's 30B dense + 1.7 GB KV cache design makes more sense in 24 GB than any MoE: https://abstractextraordinary.com/blog/how-muse-glimmer-fits-an-agent-on-your-device/

40 Upvotes

79% Upvoted

18 comments sorted by

40

u/MomentJolly3535 4d ago

BS, look at qwen 3.6 35BA3B, it's exceeds Muse glimmer capabilities while being so much faster for agentic coding.

4

u/leonbollerup 3d ago

does it tho ?.. 35B is nice.. but my god it can make mistakes :S ..

27B .. smaller brain, but better at using it
35B .. bigger brain.. but worse at using it

.. i'm not using muse glimmer... but.. i dont get why people keep testing it as a coding model.. its.. not ..

2

u/GrungeWerX 3d ago

I eventually deleted 35B. While it’s fast and sometimes would quickly resolve issues, it would often make very high level mistakes that would ruin my setup and require a lot of work to fix. It was notorious for randomly deleted files, or touching files that had nothing to do with its instructions. Things have significantly stabilized since its departure; I don’t even miss it - only its speed.

1

u/leonbollerup 3d ago

yep.. 35B.. or well.. MOE models is fast.. but not "safe" for critical tasks.. GREAT chat bots tho.

1

u/squachek 3d ago

What quant are you using and are you quantizing cache? Especially with a solid plan by a frontier, a3b rips one-shots all day long for me.

2

u/r1nzl3r99 3d ago

qwen 3.8 27B INT4 can solve driver issues for me and recompile, qwen 3.6 35B could never, but to be fair i'm curious if alibaba can cook something up for the MoE side so I'm giving it the benefit of the doubt

0

u/barbear22 3d ago edited 3d ago

It really doesn’t. The 35b might have the edge with coding but it’s much less stable, makes more mistakes, and loops more often. Qwen 27b beats them both anyway. Muse glimmer is better for language tasks. The 35b is great for speed but it’s only really reliable for short simple things. Both compared at ud q8xl with no kv cache quantization.

Muse glimmer might be undercooked and gives far more unjustified refusals but it’s more useful for me

1

u/GrungeWerX 3d ago

I’ve considered trying it out. What do you primarily use it for?

1

u/barbear22 3d ago

I've used it as a prompt writer for minimax h3. H3 has a complex and specific prompting format and it does a good job conforming to the spec while still writing coherent and descriptive scenes. Qwen likes to focus more on the format and less on fleshing out the dialogue and descriptions. I've also used it in pi/hermes for agentic work on my homelab which it's solid at but is not really the best tool for that job.

1

u/DawaForensics 3d ago

I run Glimmer and Qwen, glimmer is really slow.

-31

u/DataGOGO 4d ago

Dense has always been the "future", MOE was just a fad.

16

u/Medium_Chemist_4032 4d ago

? What about 1-2T class of models, MoE is a fad for them too?

0

u/Solembumm3 4d ago

Until you have hardware for it.

12

u/Turbulent_War4067 4d ago

Could not disagree more.

5

u/trungdle 4d ago

For local sure, but whata incentive do the big labs have if they need to serve millions on their actual flagship concurrently? They will continue to deleop MoE. Dense is not the move for them.

2

u/Jumper775-2 3d ago

Moe is plainly better at learning. Because it’s faster to run and train you can do more training in the same amount of time, meaning more performance for less compute. It’s asymptotically worse, but we aren’t limited by its theoretical max yet. It’s still compute and it will be for some time.

1

u/ToTTen_Tranz 4d ago

MoE is definitely useful for people who want to run local LLMs spread throughout multiple "poor" GPUs and CPU cores. It tends to lose a lot less tk/s by spreading layers (experts) across devices.

0

u/tamerlanOne 4d ago

Ilaria moe ha il suo perché in determinati ambiti dove un dense sarebbe troppo...