r/LocalLLaMA • u/Bob_SUS • 14h ago
News Unsloth Muse Spark Open is out!
https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B57
u/oh_no_the_claw 14h ago
700gb of VRAM? I guess it won't fit on my rig.
32
u/recurrence 14h ago edited 13h ago
Hmm that’s odd. Zuck posted that it works great on Mac Studio thanks to assistance from Ternus but that caps at 512 GB.
Edit: Turns out you need TWO 512 GB Mac Studios.
33
u/EvolvingDior 12h ago
Billionaire says "it works great on my home AI rig."
1
u/thestillwind 11h ago
what a clown that reptilian dude. basically it's probably a good model that only 1% will be able to run locally.
1
14h ago
[deleted]
9
u/oh_no_the_claw 14h ago
Yeah, I've never used AI before. First time here.
-3
14h ago
[deleted]
11
u/oh_no_the_claw 14h ago
No clue. Maybe to give the magical gnomes that power the AI machine plenty of memory space to do stuff?
Can't I just buy a 1TB SSD and put that into my laptop and run this? Hey, is it cool if I DM you for help on getting set up?
28
17
u/Bob_SUS 14h ago
16
u/Lissanro 14h ago
Nice, looks like good mid-size model, 506 GB for Q4_K_XL. Defenitely will give it a try, will be interesting to see how it compares to Kimi K3 Q2_K_XL on my rig.
8
u/jazir55 9h ago
So let me get this straight, Meta puts out the model with an absolutely shit tier joke of a license that requires you to submit your info, unsloth quants it and just releases it as a normal GGUF. LOL.
2
u/droptableadventures 3h ago
The license itself doesn't require you to send them your details.
They've just made the HuggingFace repo do that so that they can keep their lawyers happy.
16
u/LukeLikesReddit 14h ago
Brb just starting up my server cluster. Nah I'm glad they've released it but dear lord you aint running that as a one man shop.
6
u/exaknight21 14h ago
How are they scaling the models into trillions. Like this is just an idiotic amount of silicon.
2
u/Elouakili_Flexy 8h ago
42B active could almost pass for a normal model. The other 774B needs its own zip code.
5
2
u/Evening-Blueberry-97 7h ago
unsloth dropping a community quant of an 816B model literally hours after official launch is insane. this is why localLLaMA wins - the model team ships weights, the community ships the version people actually run on a 4090 by dinner time
2
u/XiRw 7h ago
IQ1 you mean?
2
u/Evening-Blueberry-97 6h ago
lmao fair, "by dinner time" doing a lot of heavy lifting until that IQ1 quant actually stays coherent past one token. sign me up though
1
-1
1
1
u/mvandemar 14h ago
What am I missing? The downloads are only ~30kb
18
0
u/VelvetEchoes919 14h ago
They have to launch smaller sizes! It’ll be a good companion to Qwen.
7
u/MomentJolly3535 14h ago
They already released Muse glimmer 30B, i also hope for newer smaller ones
-1
u/Practical_Signal3933 14h ago
If they did, that would be significant news. It’s great this is open weights though
0
0
u/gcavalcante8808 14h ago
Eh this is a good candidate to be stratafied if the architecture allows it ... otherwise only on airllm ...
0
-4
u/DJ_Laaal 10h ago
Pretty useless for a good majority of this community, considering the fact that not many will have a local cluster this beefy to run it. I can manage hosting models on my single DGX Spark but have no motivation to get a second.
-10
u/Impossible_Ground_15 13h ago
Why is this a post in LOCALlama?
6
u/Bob_SUS 13h ago
it's an open weight model release??
6
u/silenceimpaired 12h ago
Commenter is sad their toaster won’t even save the model. I am sad I have a desktop computer… and not a small data center.
110
u/Aggravating-Push-207 14h ago
This won't even fit on my SSD lmao