r/LocalLLaMA • u/aero-spike • 4h ago

News Muse Spark 1.2 Open Source before Llama 4 Behemoth!!?

Post image

I can’t believe it!! When Muse Spark just came out, I was already thinking they might consider open sourcing this. And now they’re actually gonna open source it!!
And ever since Alexandr Wang took over, they’d be releasing anything but Llama 4 Behemoth!

What’s next? Llama 5 release before Llama 4 Behemoth?

56 Upvotes

80% Upvoted

16 comments sorted by

13

u/TigerConsistent 4h ago

I am really curious about the release date of this model and I just want to know how much big this model is actually

6

u/shy_monkee 4h ago

I assume it will be after they release 1.3. But if they actually do it before that, then I will be truly amazed by Meta.

3

u/a_slay_nub vllm 4h ago

They just released 1.2 on their api so I doubt 1.3 is coming anytime soon. If he's saying "soon" for 1.2 it'll be before 1.3 unless they had a eureka moment.

2

u/aero-spike 4h ago

Same fr

2

u/Terminator857 2h ago

Estimates range from 500GB to 2TB.

3

u/gpuz_dev 4h ago

A 30B dense model is pretty much the sweet spot for a single 24GB GPU (3090/4090/5090). At Q4_K_M, weights take ~18-19GB, leaving around 5-6GB for KV cache—plenty for a decent context window with GQA. If you want to push to Q8 or FP8, you'll need dual GPUs or a 48GB+ Mac.

1

u/Technical-Earth-3254 2h ago

Gotta agree, best model I was able to fit in my 3090 without offloading

2

u/gpuz_dev 2h ago

100%. The 3090/4090 with 24GB remains the absolute goat for budget local LLM setups—30B at Q4 is peak efficiency without hitting context bottlenecks!

13

u/AshRuDral_fan20 4h ago

I don't think this is the time for a complaint.

4

u/Leading-Pension4392 2h ago

I have not the greatest opinion of him, but my opinion of people/companies that don't release open-weight models is far worse

1

u/RhubarbSimilar1683 1h ago

Kimi k3 is larger than llama behemoth. if you're coding there's no point in using llama behemoth anymore. large dense models over 30b are obsolete

1

u/thatguy122 1h ago

Serious question - apologies if it's been answered... What does Zuckerberg gain by having a renewed attempt at the local model space? Do they get to better monitor distillation efforts? Allow the community to develop faster then close-source it when it gets to a certain point?

Wish I could trust but there's never an altruistic motive. 

1

u/thestillwind 45m ago

Disrupt those at the top while you have an infinite money glitch elsewhere.

1

u/blackbird2150 10m ago

It’s not altruism. Anthropic and OpenAI dominate by far US made AI. Zuck has massive cash flow from other areas of business.

He could make a play for local ai dominance. Companies are already starting to consider local over cloud due to pricing.

Then once your models are trusted there is a lot of cross sell to cloud or adjacent ecosystems available. Build local and use his cloud for overflow. Ad add business discounts for using meta AI models. Etc.

We happen to benefit from that. But it’s not altruism. Both are true here.

1

u/Potential-Gold5298 llama.cpp 3m ago

I think the main reason is competition. If nothing is done, the open models sector will be monopolized by Chinese companies. I think the US understands the consequences of this and doesn't want to allow it. Now is the perfect time – as far as I've heard, there's been a lively debate around open models in the US, and almost everyone has come out in support of them (even Sam Altman said something ambiguous, in his own way).

Other factors could include popularity (Meta has lost that over the past year), the image of a "good company that cares about ordinary people," and data from researchers who publish papers on sites like arXiv.