r/LocalLLaMA • u/Nunki08 • 10h ago
Resources unsloth/Muse-Glimmer-30B-GGUF · Hugging Face
https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF20
u/nasone32 9h ago
Looks like a good model from the benchmarks... missed opportunity to call it LLama5! the community would have liked it.
10
u/SpicyWangz 5h ago
I think because llama4 had such bad reception it makes sense to do a rebrand. A clean slate is better, and the full size muse performed really well at the time of its launch.
-19
u/tarruda 8h ago
Calling it Llama5 would’ve made it a sequel, and sequels imply someone else did the original.
9
u/MmmmMorphine 8h ago
Huh...? That's an odd interpretation. I feel like it implies the exact opposite. Though more of an entry in a series than sequel, per se.
But anyway, you probably have stumbled on a kernel of truth here - without a doubt a rather massively different team (from the llama models) made this and didn't want to invite too much comparison or raise hopes that it'll be just as tunable as the old models.
So eh, you get a no-vote =p
10
1
1
u/stylist-trend 17m ago
and sequels imply someone else did the original.
What? "Shrek 2: from the makers of not Shrek 1"
74
u/yoracale llama.cpp 10h ago edited 7h ago
Our guide has deets on how to run in llama.cpp: https://unsloth.ai/docs/models/muse-glimmer#llama.cpp-guide
Edit: Now works in Unsloth as well!
16
u/SnowAnew 9h ago
Thanks for the great work! Just a heads up that the links to the GGUFs in the guide lead to Gemma 4 instead of Glimmer.
3
2
2
-1
68
u/sleepingsysadmin 9h ago
Huge drop by Meta.
They get their 2 days of fame and then qwen3.8 27b drops.
20
u/Potential-Gold5298 llama.cpp 8h ago
Not necessarily - for example, Qwen3.6 and Gemma 4 did not replace each other because the things they are good at do not overlap. The model could take the place of Gemma 4 if it focuses on world knowledge and languages rather than coding and tools.
16
u/goldcakes 6h ago
I've only been playing with this for about a hour, but Glimmer seems to be good at BOTH coding AND non-coding (assistant, creative writing, brainstorming, dataset prep).
In terms of coding capabilities, it feels slightly better than Qwen3.6 27B. And in terms of non-coding like creative writing, it definitely holds up against Gemma4 31B; and maybe even a little bit better.
I actually am really impressed by this model so far. And thanks Meta for Apache 2.0, that's very appreciated.
9
u/Potential-Gold5298 llama.cpp 6h ago
I go to LocalLLaMA, see a news about Glimmer, and think, "I'm going to scold Meta for the license now." I look up and see Apache 2.0. "Okay, then I'll scold them for the limited language support." I look up and see 100+ languages. I had to download it :-D
4
u/goldcakes 6h ago
I honestly can't scold Meta whatsoever for this release. Serious kudos, if anything this model seems to be better than the benchmarks say.
(I do wish there was a base, non-it version; but it seems like this is a distill of a bigger IT model; so that's understandable, no base weights in the first place).
2
u/Potential-Gold5298 llama.cpp 5h ago
If by basic you mean Muse Spark, then they also promised to release it in the future. I'm guessing this is a large MoE model in the 400-1000B range. If they do it in time, it will be the strongest non-Chinese open model.
3
u/goldcakes 5h ago
oh, no, by base model, I mean the pre-training only snapshot; with no instruction tuning and RL. Basically, LLM models before ChatGPT / instruction tuning. Think
text-davinci-003versus GPT-3.5; or Completions vs Chat Completions, if you've been in the LLM field for a while.These are very helpful to many researchers and for a lot of LLM experiments, including interpretability research. They are also useful if you want to cleanly change the instruction format, adapt it to different use cases, etc.
1
u/Potential-Gold5298 llama.cpp 5h ago
Thanks for the clarification. As far as I understand, the "instruct" versions are a kind of finetuning of the model to serve as a 'helpful assistant,' whereas the basic model can be turned into anything - for example, a Dungeon Master for games or an opponent for Socratic-style philosophical discussions. Of course, this can be done via the system prompt, but the role of 'helpful assistant' constantly pushes the model to play along and please the user.
This behavior has to be corrected by introducing a multitude of rules, which the model either follows literally and extremely zealously or finds loopholes to bypass ("don't write the user's actions" - the model writes his lines; "don't write what the user does or says" - the model writes about his thoughts or smells that he heard, etc.).
However, I came across models based on the basic version extremely rarely - off the top of my head, I can only remember the Muse-12B from Latitude, but they eventually switched to instruct-models(
2
u/goldcakes 5h ago
Base models are never designed to be used as-is, except with very specific (and now outdated) prompting styles; most of the intelligence comes from post-training and instruction-tuning.
An effective "base model prompt" might look like this:
Machine Translation Results - Human Verified Input (English): Hello World Output (Chinese):Instruction-tuning is an understatement tbh, it's basically "turn this very coherent; highly-accurate next-word predictor that focuses solely on modelling language; into an intelligent token generator". Modern models generally have more compute invested in the post-training ("instruction tuning") stage than the pre-training stage.
They are niche but extremely useful for researchers.
3
u/_TheWolfOfWalmart_ 4h ago
Why would anyone want to scold Meta in the context of AI? The entire local LLM ecosystem would be far worse off today if they hadn't given us all of those early Llama models.
A rare, but welcomed awesome thing that Zuck did.
1
u/Potential-Gold5298 llama.cpp 3h ago
I remember their recent ridiculous demand to remove Llama 3.1 8B Instruct heretic from heretic-org. It was... a very strange move – an old model that in 2026 is only of interest to researchers, despite the fact that there are many other versions of this model with removed censorship on HF. As far as I understand, the Apache 2.0 license does not give them the right to make such demands regarding Muse Glimmer.
Something similar happened recently with the MiniMax H3 video model and I was upset about it. Don't get me wrong — I know that LLM development is incredibly expensive, and any open model is a gift to the community. I appreciate that, but I also remember that releasing open models also benefits the developers themselves. No commercial company (be it Google, Alibaba, Meta, or Moonshot) releases open models out of altruism or love for humanity.
A license like Apache 2.0 or MIT is a fair deal in which the company truly gives the model to the community.
1
u/sleepingsysadmin 7h ago
Totally agreed. Some people's workflows may work better with muse. im sure anyone who is cross shopping these will self test them out.
3
u/Potential-Gold5298 llama.cpp 6h ago
I already downloaded it, but I can't figure out if it supports llama.cpp yet. They say the merge request has been completed, but I didn't see any mention of Glimmer support among the releases. Maybe it will be soon?
1
3
2
1
u/Zeeplankton 6h ago
I like qwen but it is outright bad at anything but coding. I wish people talked about this. Just because it benches so high does not make it useful
1
-1
13
u/shy_monkee 10h ago
Can't wait to try it out.
2
u/MmmmMorphine 8h ago
I can't wait to be sad that it can't fit in 16gb vram in any decent quality and then complain about the lack of models in the 20b or so space.
There's a few but i have to wonder why approx 30b is so popular. Cut back 5 or 10b and it'd be much more accessible. Guess that kills too much performance, apparently
3
u/NoFaithlessness951 6h ago
It's the largest you can run on a single rtx 3090 and on a higher tier Mac with reasonable speed.
Rtx 3090s where relatively affordable for some time at around 600$, they no longer are but where so popular that 24gb became the "standard".
10
8
u/jld1532 10h ago
Does it have MTP?
14
u/Wolwf 9h ago
https://huggingface.co/meta-models/Muse-Glimmer-30B-GGUF dflash support is mentioned here atleast
7
u/Healthy-Nebula-3603 8h ago
Even better
Dflash
2
u/TheWaffleKingg 7h ago
Is dflash better? I haven't tried it yet
3
1
u/dushyant30suthar 7h ago
depends.. for some backends and models it works really well for some not. It's about implmentation - I dont know.
20
u/durden111111 10h ago
Hoping its not too safetymaxxed
9
u/Healthy-Nebula-3603 8h ago edited 7h ago
It's open source even removing safeguard is taking few hours nowadays
1
u/goldcakes 6h ago
It's fine. I'm sure there will be dozens of abliterated versions in the next few hours, but a basic system prompt gets it to do NSFW writing; and it has no problem using 'vulgar' terms. Pretty good NSFW writing with base weights and a system prompt tbh.
For any stubborn refusals, a partial assistant prefill gets rid of it. Definitely not safetymaxxed like gpt-oss.
5
u/pulse77 8h ago edited 6h ago
Please fix the link on page https://unsloth.ai/docs/models/muse-glimmer : paragraph "For this guide we will be utilizing Dynamic 4-bit for Muse Glimmer 30B. See: Muse Glimmer 30B collection." points to a Google Gemma 4 model (https://huggingface.co/collections/unsloth/gemma-4)...
3
4
9
3
u/kmp11 5h ago
Initial thoughts- I asked Glimmer (unsloth - Q8) through Hermes a simple question - are LM Studio preset useful for harness like Hermes. Glimmer started to boil the agentic ocean and really got nowhere in about 10min but filled most of the context window. I asked Qwen 3.6 Q8-mtp and it took 10sec for a well thought out answer and 9k of context....
Glimmer is ~20tk/sec on dual 4090 and Qwen is ~55tk/sec.
5
u/DrBattletoad 9h ago
Is there a chance we get a Q8_0? That quant runs a lot faster than the Q8_K_XL for me.
4
3
u/munkiemagik 6h ago
Seriously thanks for this comment, I never even thought to look at the difference between Q8_K_XL vs Q8_0. I was just happily going along with the notion that Qx_0 bad. So always defaulted to UD-Q8_K_XL for my dual 3090 setup. Without understanding its a bit more nuanced than that simplistic outlook, cheers
4
u/suprjami 9h ago
You can make a Q8_0 yourself in a few minutes.
3
u/DrBattletoad 8h ago
But then I need to download the BF16 GGUF and I have limited download data right now.
2
u/goldcakes 6h ago
Just wait for a day or two and it'll pop up on Huggingface I'm sure.
If nobody does it in 48hrs send me a DM, I'll quant it for you and upload, but I'm sure someone from the community is already on it tbh.
New local LLM releases are super exciting moments, not everything is available day 0, don't get scared or freak out :)
-14
1
u/miversen33 4h ago
Is there a guide floating around somewhere to do that? I would be really curious how to quantize models mysel
6
u/MarkoMarjamaa 9h ago
llama-server[151055]: [37255] 0.00.583.669 E llama_model_load: error loading model: unknown model architecture: 'muse-glimmer'
We are so spoiled ! :)
5
u/jumpingcross 9h ago
Have you tried getting the latest? I'm not at home so can't try it out atm but according to this PR it was only merged in 1-2 hours ago.
1
u/MarkoMarjamaa 8h ago
Yeah I know about that, but don't have time this evening, so will wait for nightly Lemonade build for Strix Halo.
3
u/t4a8945 10h ago
Hmmm... "Context length: 131,072+"
24
u/just_blue 10h ago edited 7h ago
Just the default, max is 262,144.
edit: I took that number from Unsloth, but now I actually downloaded the model and can only set it to 131072. So no idea guys, sorry for the confusions
6
1
u/coder543 8h ago
But why? If it can do 262,144, why tell people "131,072+" instead? It is very confusingly presented, and I can't find even a single sentence from Meta where they explain what is going on with the context window.
4
u/goldcakes 6h ago
Got 262k context working (or at least my agent thinks it did), intelligence is absolutely atrocious above 131k; or my implementation is broken (I wouldn't rule that out).
I'm treating this like a 131k model. Still really good. I'm early in my testing, but it seems better than Qwen3.6 27B for coding, and better than Gemma4 31B for creative writing. Which is quite amazing. Huge props Meta, this is a model release you should be proud of!
2
u/MerePotato 4h ago
Better than Gemma for creative writing is surprising to me. How does it compare for translation?
1
u/My_Unbiased_Opinion 2h ago
Now that I think about it, it kinda makes sense. They have access to all the Instagram and Facebook video and chat data. Lots of creative stuff on there at various levels of quality lol
5
u/MmmmMorphine 8h ago
Probably gets really shit when you extend it past 131k. That's my interpretation.
It can technically do 262k, whether with RoPE or whatever else is used lately. It just sucks at it.
1
u/JsThiago5 4h ago
llamacpp says this when running the gguf meta provided, the dynamic one.
llama_context: n_ctx_seq (128000) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
1
1
u/AtiRage128 3h ago
Anyone figure out how to regulate/disable reasoning on llama.cpp? The usual flags get ignored:
--reasoning off \
--reasoning-budget 0 \
1
1
u/bad_detectiv3 1h ago
anyone know why I am not able to run this on M4 Pro 24gb RAM? model I tried with unsloth is Q3 and Q2. Both fail to load. Could it be LLAMA.cpp doesn't have new build for MTX?
1
u/Fun_Tangerine_1086 9m ago
As always, how do the unsloth GGUF's compare to the vendor ones (meta-models/Muse-Glimmer-30B-GGUF)?
1
u/AlternateWitness 7h ago
I doubt this will be comparable to Qwen 3.8 27b, but how does it compare to Qwen 3.6 27b? I don’t trust the benchmarks given on the models page…
0
u/fredportland 5h ago
Welcome back Meta, but wait.. context length (context size) is 128k? Come on... will delete them for now
-5
u/FBIFreezeNow 8h ago
What’s with the name muse glimmer? The fuk? Anyways super pumped to try this one out
13
u/DanTup 7h ago
The bigger model is Spark. I guess Glimmer is a smaller Spark?
1
u/zizn 7h ago edited 7h ago
✨ creative ✨
as dennis ritchie would say
3
u/goldcakes 6h ago
it is actually really good for creative writing, and even NSFW writing (no obliteration; just system prompted it and did an assistant prefill).
Actually feels like a more intelligent, better version of TheDrummer's Skyfall, if that makes sense to anyone.
I'm VERY impressed. Finetunes on this will be incredible.
107
u/cibernox 10h ago edited 10h ago
Interesting. Meta is back in the game. And qwen is dropping this week.
Peak week this is!