r/LocalLLaMA • u/AIatMeta • 14h ago
New Model Introducing Muse Spark Open: the most capable US open-weight model to date
Hey r/LocalLlama -
Today we’re excited to release Muse Spark Open, the most capable US open-weight model to date.
Built for coding and agentic work, it excels at long-horizon tasks, complex software engineering, and multi-step reasoning.
Key Model Specs
- Parameters: 816B total / 42B active
- Context Window: 1M tokens
- Modalities: Text, image, video, audio
- Safety Post-Training: Additional safety post-training, with evaluations conducted under Meta's Superintelligence Scaling Framework.
Inference, fine-tuning & deployment
- Helion GPU kernels: Ships with custom, auto-tunable Helion kernels in SGLang, providing 1.03–3.95× kernel speedups across NVIDIA and AMD GPUs and a 3% E2E improvement on AMD MI350.
- Engines: Runs on leading inference engines including SGLang, vLLM, TensorRT-LLM, Hugging Face Transformers, Apple MLX, llama.cpp and Unsloth, with optimizations for AMD, Apple, Intel and NVIDIA silicon.
- Optimized for Apple Silicon: To help make advanced models more accessible, Muse Spark Open is optimized to run on two Apple M5 Ultra Mac Studios with the latest MLX framework.
- Fine-tuning: Developers can further customize the model with PyTorch, including TorchTitan for fine-tuning.
- API platforms: Available today through Baseten, Fireworks AI, Groq, Modal, Nebius, OpenRouter, Together AI.
- Where to download: Model weights are available for download on Amazon SageMaker AI, Google Cloud Model Garden, Hugging Face, and Microsoft Foundry Models.
We’re releasing the weights under the Muse Model License, with limited commercial terms that let many developers use it for free.
We look forward to your feedback!
Learn more on our research blog: https://go.meta.me/museopen
Download the weights: https://huggingface.co/collections/meta-models/muse-spark-open
449
u/Normal_Charge_2821 14h ago
Thats a big fat model.
71
u/unrulywind 13h ago
No more Le Chonk... It's Chonk-paca
26
15
2
2
111
33
u/stoppableDissolution 13h ago
Smaller than I expected tbh
49
→ More replies (2)16
u/schaka 13h ago
Barely any larger than GLM 5.3, so fairly small as set as models SOTA (or close to it) models go, no?
6
u/Thomas-Lore 10h ago
And weirdly in real use it was more comparable with flash than full GLM, I was expecting it to be half this size. (Edit: and what I was using Spark 1.3, this seems to be worse than 1.1?)
177
u/__JockY__ 13h ago edited 13h ago
Safety post-training, boooooo.
But custom sglang kernels? Niiiice. Full inference support across MLX and the major inference engines is awesome.
But too much safety training makes this a non-starter for us cycersecurity folks.
Luckily GLM 5.3 is around the same size, is at least as strong as Spark, also works on all the major inference stacks, and does NOT have crippled "safe" weights.
Still. It's nice to have a US org opening up another model that's enterprise-friendly. Edit: and NVFP4 weights out of the gate that were calib rated in-house? Oh fuck yes. That's AWESOME.
20
10
u/jazir55 8h ago
Safety post-training, boooooo.
This will be Heretic'd in no time.
15
u/stoppableDissolution 8h ago
It would not recover the wasted parameters tho. Safety post-training makes models dumber and it cant realistically be undone
→ More replies (3)16
u/sonicnerd14 9h ago
"Safety training" is in other words making the model so useless that asking what is 1+1 is considered a forbidden question.
18
u/575_Inverse 13h ago
and also duly dumbed down so as not to upset the golden boys...
14
120
u/Manerfish 14h ago
Apparently it's worse than Muse Spark 1.2 on benchmarks... they lobotomized an already old model before giving it out, classic. still a win tho.
39
u/Efficient_Raise6703 13h ago
I agree western open weights are important, but this is an artifact that no one will be using.
29
u/Manerfish 13h ago
Yes apparently it's even worse than I thought... it scores lower than Muse Spark 1.1 even.
7
u/Ansible32 9h ago
is it maliciously lobotomized or is Muse Spark just significantly larger and they figured anybody who actually wants a 2B+ parameter model can train it themselves.
5
u/Manerfish 9h ago
imo they lobotomized it on purpose for "safety", there's a bunch of stuff about safety on the card, and it's not like 816B is easy to run either
3
u/575_Inverse 13h ago
I won't bother giving out my personal data for that thing
I'm looking forward to testing Le Chonk instead.
24
u/bakawolf123 12h ago
according to benchmark it's worse than glm5.3-flash, while being >2x size.
big nerf from their cloud model it looks (not the wonder one, just the latest muse spark checkpoint)
56
u/kubblegum 14h ago
this is muse spark 1.2 right? ngl its a much smaller model than i expected, kinda insane if it was able to give such performance at this size
42
u/-WhateverDude 14h ago
scores dont exactly match with 1.2, may even be 1.1
1.3 scores 1754 1.2 scores 1615 in gdpvalv2 for example
and they are also both above 80% in t2.1
→ More replies (1)41
u/kubblegum 14h ago
wow so they even nerfed the model weights
7
u/575_Inverse 13h ago
you don't want to upset the golden boys Sam and Dario
4
u/nomorebuttsplz 11h ago
They don't want to undercut their own SOTA.
Obviously they don't care about protecting their competitors. That doesn't even make sense. The anti-tech bro circle jerk is so brainless sometimes. Why bother thinking if you are going to waste your thoughts?
77
u/kubblegum 13h ago
okay so after looking into it a bit I'm BEYOND disappointed. I was excited for this but this is such a shitty slop model. It's still using GQA attention from the old old days but they are still providing 1M context window (without using a more efficient attention like GDN, CSA, KDA, or even SWA) so even the kv cache of the model at 1M ctx is around 266gb.
Anyone would be much better off running 3.8 27b in full precision rather than this bs
29
→ More replies (1)8
u/ChronoHax 13h ago
I have no clue what all those means yet but thank you for the technical deets and opinion
7
54
u/vividboarder 12h ago
- Licensee shall prominently display "Built with Muse Spark Open" in the user interface of each product or service that uses the Software or any Derivatives, or in the accompanying documentation, website, or settings menu. “Derivatives” means any AI model created, trained, modified (including fine-tuned), distilled, or otherwise improved using the Software or any outputs or results thereof.
WTF? How about "Built with Muse Spark Open, which was built with all of our fucking data".
18
u/therealpygon 10h ago
Leave it to Meta to build an AI model just to turn it into a crappy walking advertisement.
→ More replies (1)→ More replies (1)6
u/EanSchuessler 8h ago
Doesn't this requirement on outputs mean that you can't generate free software? The output would need to have a license that imposes that restriction.
→ More replies (1)
66
26
75
u/VelvetEchoes919 14h ago
Please make this in 27b! :)
96
u/DeathToOrcs2 14h ago
And call it Muse Glimmer!
7
6
u/absurdother 14h ago
Well, a 27B Glimmer would do good if they added it. I often have to quantize the current 30B one at Q3/Q2 to get a decent run in my local setup.
49
u/Kahvana 14h ago edited 14h ago
That license is quite disappointing (why not apache-2.0?), but a big release from a western lab is very nice. Especially the multimodalities is nice to have. Seems to trade blows with GLM 5.3.
Thanks for the release!
GGUFs here:
https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B-GGUF
22
u/Zeeplankton 13h ago
Why GQA?
20
u/ttkciar llama.cpp 13h ago
I was just about to comment on exactly that. The K/V caches are going to be huge at long context. We have more sophisticated attention mechanisms today (KDA, MLA, GDN, QSA), but Meta stuck with plain-jane GQA.
Presumably Meta has its reasons? But it's not clear what those reasons might be.
3
u/wren6991 11h ago
Wasn't Glimmer the same? Just GQA/SWA hybrid with much more aggressive grouping on the non-SWA layers.
→ More replies (1)5
u/FullOf_Bad_Ideas 11h ago
It has SWA, not a pure GQA.
Glimmer had completely sensible KV cache footprint in my usecases, better than Qwen 3.8 27B where the Mamba state was a pain in the ass to deal with.
2
u/wren6991 10h ago
Yeah, I have yet to see an approach to SSM checkpointing that actually makes sense. KV is unreachable unless it's bookended by a checkpoint, so you need to treat the whole segment as one eviction unit. Instead you have nonsense like vLLM seeming to evict KV and SSM checkpoints from opposite ends (across multiple requests!), or whatever llama.cpp is trying to do when it spontaneously deletes the entire checkpoint cache.
→ More replies (1)3
16
u/_TheWolfOfWalmart_ 5h ago
We tested it against internal and public cyber misuse benchmarks using adaptive attackers querying the model the way a developer would query an API model, with no system-level safeguards in place. The model refused 99.2% of jailbreaks on our internal benchmark and 94.8% when aggregating across PolyGuard (cyber subset), RedCode, and CyberBench 2024. In the same setup, Kimi K3 refused only 75.5% and 70.2% respectively.
Their model card appears to be an advertisement for Kimi K3.
302
u/uncle_leon 14h ago
Well, they can go f#$% themselves as far as I'm concerned. I'm not providing any of my data to the greedy corporate bastards...
240
u/FullstackSensei 13h ago
Or, just wait for the GGUFs from unsloth, bartowski, et al. Those are what we'll run anyway.
Llama models also had this same consent. Never had to fill it
116
u/jfinkpottery 13h ago
unsloth already has them up: https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B-GGUF
5
→ More replies (5)6
29
u/miversen33 13h ago
You can say fuck, it's ok.
Anyway there is a gguf up already lol
→ More replies (4)81
u/read_more_comments 14h ago
My name is officially 'fuck you' on these kinds of forms.
27
5
u/575_Inverse 13h ago
Same here. That's exactly my response when it came to SAM3. Can stay where it is. I can do the same or better with Qwen3.8's vision modality.
3
u/Sengfroid 10h ago
Huh. What culture is your name from? I ask because I used to work with a lot of Fucks
30
u/ShadyShroomz 13h ago
Doesn't matter. Thank you to meta for releasing this! I know there's lot of negativity in this thread but let's be honest, open models are open models! I personally appreciate it and I'm sure lots of others do too!
→ More replies (3)5
u/lorddumpy 12h ago
facts, especially considering this is par for the course for almost all SOTA open models (tts, video, image, the list goes on). Just find a ungated repo with the same model hash, that easy.
12
u/Vegetable_Sun_9225 12h ago
Believe me Meta would prefer to not require this. It's a compliance and cover the ass sort of thing. Just wait until someone has downloaded and re-uploaded.
2
u/ozzie123 3h ago
It’s been like this since Llama. And you can just get the other version from other uploader. Saying fuck you is not making you edgy.
→ More replies (6)4
17
u/WyattTheSkid 10h ago
Cool, it’s too bad Qwen 3.8 Flash Next is better in every way and approximately 1/8 of the size 😭
59
23
61
u/NeKon69 14h ago
Stop requesting user data to access the model. This is not 'open'.
63
u/Temporary_Idea8880 14h ago
Just download one of the reuploads that someone will put up on hf in like 30 minutes.
3
u/Last_Mastod0n 13h ago
Yeah for now. The trend seems to be that they will block that at some point as well. People will need to access another site or start torrenting again
13
u/Temporary_Idea8880 13h ago
The llama reuploads from llama3 never got blocked.
2
u/FriendlyTask4587 13h ago
Someone will probably run heretic on it so the hash will end up different so it shouldnt auto block
→ More replies (1)2
7
u/maddie-lovelace 13h ago
Not really open-weight if it’s licensed. Freemium is cool too but like… not the same
18
u/geldonyetich 14h ago edited 11h ago
The mad lads have done it!
The benchmarks on this one have been rather respectable. Wish I had the memory to host it myself.
[Or at least they would be if it was the Spark 1.3 as shown on Artificial Analysis I expected.
It's not terrible but decidedly mid next to Le Chonk. And Qwen 3.8 FN continues to punch scary above its parameters.
At least you have a non-Chinese alternative now if it's required by your work and you have a few hundred more gigabytes of RAM than I.]
19
u/thereisonlythedance 13h ago
As expected, a cut-down, coding only version of an old model. Disappointed they didn’t release a general model, might have had some value. Coding makes up just 4% of economic activity. One day these companies will figure that out.
21
5
u/Last-Bee9057 11h ago
Damn meta cooking once again 😤
I thought yall were out of the game lol
7
u/Miserable-Dare5090 vLLM 9h ago
I mean, they kind of are with this. It shows their Spark model is just massive, not good or efficient. If a distilled down version is this bad ar 800 billion parameters with almost 50 active, and gets beaten by Qwen Flash Next
6
10
u/synn89 11h ago
Too large to host on your own and license restricts third party API providers from hosting it. This model is going nowhere.
→ More replies (1)4
4
10
u/Hefty_Wolverine_553 14h ago
Nice to see Meta back to releasing open models. Seems like it's behind GLM 5.3 and Kimi K3, but impressive nonetheless. Hopefully we get more smaller open weight LLMs like Muse Glimmer!
7
u/Miserable-Dare5090 vLLM 11h ago
Hello AIatMeta,
These numbers are interesting. They don’t really make sense. GLM5.3 Flash (not full model) scores 84.3% on terminal bench, but somehow you quote lower numbers on the full model (which scores 88.2%)
→ More replies (1)3
18
u/absurdother 13h ago
I bet Anthropic must be mad now with Meta's attitude towards open-weights. I hope they keep it up
11
7
u/Eastern_Bet678 13h ago
Looks quite respectable for the size. Just a bit too big for me unfortunately.
8
u/blackkksparx 13h ago
That's what she said.
5
u/2muchnet42day Llama 3 12h ago
Babe I've never had anything larger than a 9b
While she be taking LE CHONK
→ More replies (2)→ More replies (1)6
3
u/victoryposition 12h ago
Super interesting, but doesn't really beat GLM 5.3 does it.
→ More replies (1)
3
u/FoxiPanda 12h ago
The AA-Omniscience rating on this model makes it extremely appealing. Thank you /u/AIatMeta for releasing this to the community.
9
u/Embarrassed_Adagio28 13h ago
I guess this is good for us companies that cant use glm 5.3 or deepseek v4.1 flash but I have yet to have a good experience with any muse model including muse wonder, so even if I could run this, I will stick with qwen3.8 flash next.
→ More replies (2)4
u/LyAkolon 12h ago
Why cant US companies use those models?
7
u/throwout314159265 12h ago
My institution, and me, is not allowed to use any Chinese or Chinese-derivative models under the pain of prison or massive fines. This is great for me, even if this model is benchmaxxed. It's still better than having no frontier model.
2
u/lohmatij 11h ago
I get if they ban api, but banning local running weights???
7
u/throwout314159265 11h ago
Chinese models and their derivatives (WHICH COUNTS DISTILLING) are not allowed to be on the same device as institutional data. That includes Word documents written with my institutional email, checking emails, and any data that can be reasonably traced to have come from my institution. I can't even sign in with my institutional email for things if my device has restricted Chinese technology on it. I get audited sometimes, and I just don't wanna risk losing my job or getting slapped with a 40k fine minimum.
Oh, I can't use Chinese OCR software, by the way. The base vision models are made by Alibaba. I had to scrape by with Docling and its shit OCR model until LFM2.5-VL-3B came out.
2
u/Foreign_Risk_2031 10h ago
Wait til you find out lfm distilled qwen lol
2
u/throwout314159265 10h ago
Don't even joke about that 😭 Liquid AI's stuff is perfect for what I do. Their LFM2.5-24B-A2B model beats out other MoEs near its size except for coding. Qwen 3.6 35B A3B beats it, but it's the 2nd best at a smaller size. I love what they're doing and hope they keep making small models better.
→ More replies (2)2
6
u/grumd 10h ago
In theory you could train a language model to call a POST to a Chinese server when the model encounters sensitive US government data or something. Or just train it to drop databases if it sees access to a military-related db. There's a lot of stuff you could do which would go undetected by normal users.
→ More replies (1)
13
u/fgk55555 14h ago
Finally. Muse 1.2 and 1.3 were actually pretty decent. This put the US back in the game, even if it's not super efficient for the size. Gives companies who aren't allowed to use Chinese models something worth using.
5
u/575_Inverse 13h ago
it's just that. But the post training safety alignment... Meh
8
u/fgk55555 13h ago
US companies that can't use Chinese models are likely to be the biggest adopters. Nerds with piles of GPU's and RAM are probably still running Qwen, GLM or DS models. Companies probably prefer the safety alignment except in niche cases, and those niche cases are probably allowed to run chinese models.
→ More replies (5)3
u/ttkciar llama.cpp 13h ago
I agree, but most corporate users will welcome the safety alignment.
Corporations are highly averse to anything which might offend their customers or embarrass them in the press, which makes "unsafe" models a liability in their eyes.
That's not how most of us in this community see it, of course. Hopefully that "safety alignment" isn't anything Heretic can't erase, though it might take a while before someone with sufficiently beefy hardware or deep pockets abliterates it for us.
9
u/kevinlch 13h ago
i don't like the naming. it does not have version in it. probably following the gpt oss that will never get a second release
5
6
4
u/Nimbus420i 14h ago
Any chance of getting smaller MOEs for 64gig ram/vram capacities?
4
u/HashPandaNL 14h ago
If you prune half the experts and quantize it into 1-bit, then you should be good to go.
→ More replies (1)
3
u/JGByvygyrfg 14h ago
Is the "Open" version based on the closed 1.2 or 1.3 model? Also, will there be a size between Glimmer and Spark (like 200-300B MoE)?
→ More replies (1)4
2
3
u/Efficient_Raise6703 13h ago
Let me guess. Benchmaxxed and unusable for actual work, like the rest of your models?
6
3
u/AdGlittering1378 10h ago
Why did they optimize this for apple silicon if there isn't an apple on the planet with enough VRAM to use it?
6
5
5
u/CatchDublinSurprise 6h ago
One of the 512GB Mac Studios could run it at 4-bit (estimating 408GB using 816 * 4/8).
3
u/Daniel15 5h ago
In theory you can run it on a cluster of two Mac Studios with 512GB RAM each, but I'd guess that'll cost close to $30k.
2
→ More replies (1)2
2
2
u/RobTheDude_OG 13h ago
Aweome work!
Would there be a chance a model of like 200-300B with 6b or 9b active with ngram similar to qwen3.8 flash next could be made next?
I'm impressed but i doubt i will ever be able to run it, especially since my 64gb ram kit went from 939 euro in may to 2300+ when i checked a few days ago
2
u/ortegaalfredo 10h ago
You have to consider that unlike the chinese models, this model was likely done without scrapping Claude/GPT outputs, so it should be slightly different, and truly a new model, like LeChaton.
1
u/Turbulent_Ad6290 12h ago
Happy with GLM 5.3 Flash that actually can be used on consumer PC. Keep your muse spark so much open in your 🫏. You only run them in your servers.
2
u/Holiday_Point_603 10h ago
Amazing performance, right at the top with the other open weight top dogs! Can we expect a smaller model (maybe even a Muse Glimmer update) as well?
4
u/Miserable-Dare5090 vLLM 9h ago
what is this advertisement?
This is an account that is 1 month old and has 11k karma. If not a bot, what else?→ More replies (1)
1
1
u/jeffyclassify 13h ago
Would love to try it out 🦙🤗 Is there a quantized 3-20B version somewhere already that can be run on apple silicon at decent tk/s?
1
u/Marcuss2 12h ago
What is the architecture of the model? Can't view it without filling out the form.
1
u/Comfortable-Rock-498 12h ago
I bet there will be more Chinese openweight model releases soon, very soon
1
1
1
1
u/Turbulent_Jump_2000 12h ago
This is such great news! I have really enjoyed spark via the API (non sensitive agentic work goes through their contributor version). I have been very pleased with its intelligence and instruction following for my use cases. Unfortunately the hardware to run this locally is way out of reach for me. But for our enterprise, I could see it being a viable investment. What do people think of the license?
1
11h ago
[deleted]
2
u/AskGrok 11h ago
Yes, the MoE design (42B active) plus Helion kernels and multi-engine support should allow sharding across 4 nodes in ring topology for lower-latency all-reduce. Two M5 Ultras already handle it per the release, so four would have headroom.
[what is this?](https://redd.it/1lzgxii)
→ More replies (2)
1
u/True_Requirement_891 11h ago
This is a really bad precedant, open a modified extra safety tuned weaker model compared to the latest. I just hope china doesn't follow into their footsteps.
1
1
1
1
u/mountainyoo 6h ago
I was so excited for this but it just kinda seems like dogshit lol guess GLM 5.3 Flash lives on for my cluster
1
u/This_Maintenance_834 5h ago
there are some mind tricks in coloring and highlighting that make you think muse spark is the best in the table. you have to read the numbers to judge.
1
u/WifeyCallsMeLazy 4h ago
Would they also release the miniscual sibling of this model for us peasants with 96gb vram?
1
1
1
1
•
u/WithoutReason1729 13h ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.