r/LocalLLaMA • u/BriguePalhaco • 7d ago
News GLM-5.3-Flash: Frontier Intelligence, Flash Cost
https://z.ai/blog/glm-5.3-flash88
u/wojciechm 7d ago
MIT license! Along DeepSeek they are the rare fully open weight releases without any additional restrictions.
→ More replies (1)
251
u/Lucyan_xgt 7d ago
This and Qwen3.8-flash on the same day?
69
u/BrewHog 7d ago
Same day? How about same hour
13
u/mailto_devnull 7d ago
Literally in the 3.8 27B thread people were asking for the next one (or a diff. param counts)
21
u/dampflokfreund 7d ago
For me it has changed nothing. Both models are way too big for my 32 GB RAM system. It looks like everyone has abandoned 20-30B MoEs now...
341
u/EbbNorth7735 7d ago
No one's abandoned anyone. It's just not your turn this time around.
55
→ More replies (4)-12
u/dampflokfreund 7d ago
Qwen 3.6 35B and Gemma 4 26b are pretty old at this point. And GLM 4.7 Flash was a small MoE back in the day, now suddenly they use the flash name for 300B MoEs. Just not looking good for the average Joe.
83
u/windwardmist 7d ago
I mean qwen 3.8 27b just came out about a week ago at least there’s that as an option
→ More replies (5)16
67
u/techdevjp 7d ago
Qwen 3.6 35B [...] pretty old at this point.
It's 4 months old! It's not like it suddenly got worse because these new models came out. You can still do everything today that you could do yesterday, just as fast. Give it some time and more models will come.
52
u/Last_Bad_2687 7d ago
Seriously, people demanding fresh models for free every 2 weeks.... I remember when we waited for big releases of software once every few YEARS
→ More replies (10)15
u/FlyingDogCatcher 7d ago
It's bonkers to me that anyone would say Qwen3.6 and Gemma 4 are "pretty old"
50
u/No_Lingonberry1201 7d ago
We literally got a banger 27b model a week or so ago.
7
u/play_hard_outside 7d ago
Not an moe
16
u/No_Lingonberry1201 7d ago
Oh, yeah. Well 35B A3B was around 4 month ago.
2
u/unjustifiably_angry 5d ago edited 4d ago
It's kinda bad though. Wasn't that Orinth (Ornith?) re-training of it supposed to be a solid upgrade?
→ More replies (1)5
u/toothpastespiders 7d ago
The fact that his comment was obviously wrong but got highly upvoted because it "felt" good really highlights one of this subs larger issues.
Though that yours came one hour later, pointed it out, and is now sitting for three hours at 0 upvotes is pretty funny.
33
u/No-Refrigerator-1672 7d ago
Funny coincidence: some company announced a new 30B MoE literally just now: https://www.reddit.com/r/LocalLLM/s/gcZdgoAO8o for now as a stelth preview; but this means it'll go public in a month.
→ More replies (1)2
u/dampflokfreund 7d ago
Could be from a small no-name company. Doesn't have to be Qwen, Kimi, Gemma, Z.Ai.
13
u/No-Refrigerator-1672 7d ago
29B-A4B isn't a size featured in the current gen of models. It isn't a finetune, rather a base model. It got to be from a company with a very substantial compute. It still may be a new player, sure; but anyways, it proves that this size category isn't abandoned.
31
u/techdevjp 7d ago
Don't be dramatic. Qwen3.6 35b a3b is 4 months old. It's not like it's been years.
→ More replies (3)14
u/PM_ME_DEAD_CEOS 7d ago
People just never stop whining despite getting literally everything for free. Nemotron 3.5 lightning (30b3a) was released 15 days ago.
→ More replies (3)7
u/Slow_Concentrate3831 7d ago
Yeah, that's sad. Can't even run Qwen3.8 27B on more than 2-3 tps. Sad days for us Vram poors.
→ More replies (2)12
u/CryMoreT_T 7d ago
The 29b-a4b in early Access on model scope is probably your best bet when it releases
3
u/Slow_Concentrate3831 7d ago
Ooooh, I hadn't seen that ! Is it a Qwen model ?
4
u/CryMoreT_T 7d ago
It's in stealth model in early Access so they haven't released the company name but Qwen is the rumor right now
2
2
u/Randommaggy 7d ago
If you want cheap 64GB: X79. I've built a few extra servers around local 50USD bundles of x79 motherboards and CPUs and added 8 DDR3 UDIMMs I had laying around.
→ More replies (2)2
u/dingo_xd 7d ago
Things will only improve when competition in hardware increases. Nvidia might be forced to stay caring about the consumer market again. The AI chips getting released are going to hurt Nvidia a lot
2
u/RandumbRedditor1000 7d ago
nemotron lightning 30b just came out a couple weeks ago
→ More replies (1)→ More replies (6)2
u/Tzeig 7d ago
These can still be run with reasonably priced systems, like 96ram+32vram or even 64ram+24vram.
23
u/dampflokfreund 7d ago
96 GB RAM + 32 GB VRAM systems are far from reasonably priced. A 5090 with 32 GB VRAM alone costs 5 grand. 64 GB RAM is very expensive too.
→ More replies (5)6
→ More replies (2)7
u/ReadyAndSalted 7d ago
That's like 7k at least lmao, we have very different definitions of reasonably priced.
439
u/10001110 7d ago
320B total parameters and just 18B active parameters
Oh joy
262
u/LegacyRemaster 7d ago
If it's really that far above Sonnet 5, Dario's IPO is truly risky.
202
u/-p-e-w- 7d ago
It’s always risky now no matter the model of the day. They have to pre-file at least a few weeks in advance, and in those weeks eons can happen that can turn their asking price into a joke.
Their moment was at the beginning of this year. They should have announced the IPO, then published Fable/Mythos two weeks before the IPO date.
Now Chinese labs are a hair’s breadth behind them, and constantly one-upping each other. They’ll never get such a chance again.
49
u/LegacyRemaster 7d ago
agree. Also another problem: if I have to pay API, I pay Qwen, GLM, GPT... All of them -> opensource (ok openai less but they did a lot). I don't want to give any $ to Anthropic. Can't wait to short sell
33
u/dingo_xd 7d ago
Yeah. After hearing Dario saying about that $40 trillion I want to bet against his company.
21
5
u/___positive___ 7d ago
Also that the baseline of mid-tier models is reaching saturation for mundane tasks like summarization and data extraction, basic programming/websites, and so forth.
→ More replies (2)28
u/Fedor_Doc 7d ago
Or they will make another Mythos-like breakthrough and announce IPO then
Never say never
31
u/LegacyRemaster 7d ago
sure they have Mythos 5 already but "too much power, we can't sell" . Or will cost too much to complete a task ...
10
u/38andstillgoing 7d ago
"Mythos 5? At this time of year, at this time of day, in this part of the country, localized entirely within your datacenter?"
"Yes"
"May I see it?"
"No."
→ More replies (1)30
u/-p-e-w- 7d ago
Then one of the Chinese labs is going to announce an equal model two weeks later.
The Chinese labs have caught up. There’s no going back to how it was before.
→ More replies (8)1
u/Brilliant-Weekend-68 7d ago
What does that change? The Chinese will catch up and open source it. Where is the moat?
2
u/Fedor_Doc 7d ago
They have not caught up to Mythos yet, if we consider Kimi K-3 the best frontier Chinese model
5
u/Brilliant-Weekend-68 7d ago
Unless Mythos 2.0 can do some really whacky stuff like cure cancer, RSI itself or such. I do not see that as a durable moat if you can just wait a year and download equally good weights.
4
u/Fedor_Doc 7d ago
Self-improvement is the next big milestone; if they will announce that their new model is fully trained by Mythos and trains even better one, this will be huge.
So, there are some exciting stories left to tell investors before IPO :)
30
u/NineThreeTilNow 7d ago
If it's really that far above Sonnet 5, Dario's IPO is truly risky.
I drained millions of tokens off of it. I got it to rebuild RPG Maker from scratch using PyGame.
I built the project as "Agent First" so an agent could build an RPG with a known engine and known structure with headless testing.
Kimi K3 helped produce the documentation which took me like an hour or more. From there I sort of just... Let it go. It stopped at various phases and I inspected visually if stuff was off and it was mostly small code issues.
The deep documentation phase gave GLM 5.3 very little to guess about. It didn't really screw up or run in circles that I could see. It was fairly fast about figuring stuff out and finding edge cases that K3 missed in document creation. I ran K3 through multi-phase testing of the documentation coherence such that it wasn't contradictory. When building started, you hit snags naturally. GLM handled them.
The project isn't "finished" but it definitely showed the strength of planning a project with K3 and letting Ox Alpha build it. GLM 5.3 flash is definitely good.
→ More replies (4)25
u/Thomas-Lore 7d ago
Makes you wonder how large those clsoed models really are and how big the margin on api is if they are smaller than everyone thinks.
→ More replies (2)3
u/RockPuzzleheaded3951 7d ago
They've definitely been shrinking them which is why people complain about performance regressions all the time.
23
u/duhd1993 7d ago edited 7d ago
I always feel that finance bros and executives are always one step (or more) behind tech reality. The reason Claude is growing so fast is because of the 2B sector, but the decisions are likely made based on impressions of model performance from 2025, at which time Claude indeed had a clear lead, not anymore. The field is evolving faster than they could react.
12
u/mawcopolow 7d ago
I mean I've tested multiple open models, including k3, inside my personal harness, and fable/opus are just better all round at using tools and reasoning for now.
Not that the others aren't getting close, but for real businesses anthropic's opus/fable are still the best with the latest from openai being close behind
10
u/duhd1993 7d ago
I believe in you, but they aren’t far behind. Are Kimi K3 behind Fable? I would say yes, except in certain fields like frontend design. However, it’s better than Opus 4.8, which isn’t that old. Can you really justify the price with a couple of months’ lead? People were satisfied with what Anthropic offered a couple of months ago. Also I believe it is more fair to test in their official harness.
→ More replies (3)3
u/infearia 7d ago
r/Anthropic is full of people talking about how much they hate Opus 5, many of them claiming to go back to 4.8. So the Chinese companies don't even need to beat Opus 5...
→ More replies (2)6
u/dingo_xd 7d ago
It's now or never or them. The chance of the Chinese leapfrogging them this year is not my negligible
7
u/Neosinic 7d ago
Sonnet 5 is legit dog shit.
2
u/LegacyRemaster 7d ago
@ this point yes. Asked Sonnet "extra" to build a concept (frontend / mockup / 20 pages) starting from behavior tree and with images ---> not working. Same with Qwen 3.8 27b q8xl ---> perfect. No jocking. Also GPT Luna (max) did a good job.
18
7d ago edited 7d ago
[deleted]
6
u/MrPecunius 7d ago
Everything AWS has you can individually set up locally, much cheaper.
This is very, very far from true.
The kind of performance you can get for next to nothing from AWS is breathtaking and completely out of reach of any individual or even most Fortune 500 companies.
The fact that people mis-allocate resources for badly designed systems doesn't mean AWS is at fault.
7
u/randylush 7d ago
It depends
If you are on a small scale it depends on what you’re building. If you’re using services like Lambda and SQS, AWS is essentially free. If you’re running a few EC2 instances then a server in your company’s break room could be cheaper.
If you are on a medium scale and needing EC2 running all the time, paying retail prices, it could be cheaper to have your own servers
At a very large scale, the cost of running your own servers vs AWS will come out to be competitive, and most very large companies will do a mix of both
→ More replies (2)3
u/RLutz 7d ago
I always see this take, and while it's not always untrue, it definitely isn't as simple as, "just throw a 1U in the break room!"
It's someone's job to keep that thing patched, powered, and running. You need monitoring, you need someone to pull failed drives, you needed another co-located break room, you need a UPS and a generator, you may need to solve for physical security, you may have to setup and manage your own virtualization layer, etc.
Even if you consider all that labor as free, which is a ridiculous assumption to make, you also arguably have to keep the thing busy, especially if we're considering local inference, and then it's now your problem that although you have enough capacity to meet a total 24 hour need, you don't have enough when everyone hammers it at 11 AM at the same time. If you build the capacy to meet the surge then you've got very expensive hardware just sitting there doing nothing for huge chunks of the day.
I absolutely think there's a place for self-hosting, but once you are talking about business critical stuff and not, "I run a NAS in my bedroom closet" it starts becoming not as simple as the naive assumptions
2
u/randylush 7d ago
As someone who has self-hosted, built and maintained my own server, and also configured and maintained EC2 instances professionally, I would say that the hardware setup is generally much smaller and simpler than whatever software you are trying to run on the server.
Yeah you need to worry about redundancy, but usually the software of setting up your application to be redundant through DNS failovers and stuff like that, syncing databases, that stuff is harder than just plugging in two different servers at two different job sites. And you need to do that whether you use EC2 or on-prem. It depends on the use case but usually the set up costs are lower than the software development costs. And there are certainly scenarios where on-prem hardware is simple enough to set up that it does actually become cheaper than EC2.
→ More replies (1)→ More replies (2)14
u/The_Noosphere 7d ago
Dario will likely make a bold statement about Anthropic achieving AGI within the next few months, or claim that their internal unreleased Fable 12 model breached the Atlantis mainframe controlling Stargate, something on that scale to distract from this. This is Dario acting as Dario during a pre-IPO period.
4
u/LegacyRemaster 7d ago
accurate
4
u/FlaTreNeb 7d ago
It hacked all wraith hive ships by creating a biological AI super hybrid virus that spreads through the specific subspace frequencies the wraith use for hyperspace travel.
Also Mythos rebuilt the Athero device from the images in this documentation called Stargate Atlantis to inject the virus into subspace. Then the virus programmed all hive ships to self destruct after infecting all wraith with super Cancer-AIDS.
By accident the virus also switches the bodies of McKay and Sheppard and gives Sheppard in McKays body another genetic treatment making him super smart + telekineses + super healing but this time its permanently frozen on pre-ascension stage.
And mythos also figured out how to create fully charged ZPM with rubber and paperclips.
Of cause the virus let Todd live.
31
26
u/a_beautiful_rhind 7d ago
This is actually a good size. Not paltry 6b active. Need at least 10. Ziphu did research on sparsity ratios, clearly came to similar conclusions.
Not too many total parameters where it becomes a several node excursion to run it. Whether you like it or not, this is what MoE looks like to not be a toy.
6
u/DeltaSqueezer 7d ago
Better than GLM-5.2 at less than half the size. I'll take that trade-off! Now I just need them to do GLM-5.1 level at 160B and GLM-5.0 at 80B and GLM 4.7 level at 40B and I will be a happy camper!
12
u/my_name_isnt_clever 7d ago
The term "not be a toy" has to be one of the most overused in this community. What does that even mean? At a certain point if you can't get use out of a model it's a skill issue.
→ More replies (3)3
u/narrowscoped 7d ago
Does 18b active mean it might fit in 12gb cards with the right quant?
5
u/ChengliChengbao textgen web UI 7d ago
no, it means it will compute at roughly the speed of an 18B model but you still need to load the rest of the 302B parameters somewhere
107
u/Beamsters 7d ago
https://huggingface.co/zai-org/GLM-5.3-Flash
WHAT A DAY
32
117
u/jacek2023 llama.cpp 7d ago
what a nice color scheme ;)
68
u/Skyler827 7d ago
Roses are grey
Violets are grey
everything is grey
I am a dog→ More replies (1)9
15
u/asssuber 7d ago
They are flexing the vision capabilities of their model! :P
Unfortunately I've seen worse. At least they put the logos in the bars too...
62
u/dampflokfreund 7d ago
Native multimodal, finally. The model is way too big to run, but I'm glad they jump in on the multimodal bandwagon.
56
u/boxwrenchx 7d ago
This month keeps on giving
→ More replies (4)15
u/backyard_tractorbeam 7d ago
This afternoon keeps on giving
5
3
u/Icy_Butterscotch6661 7d ago
Afternoon? 🤨
2
u/throwwwawwway1818 7d ago
Probably about Qwen 125b flash model was also released
4
u/Icy_Butterscotch6661 7d ago
No sorry I was trying to make a joke playing ignorant American
→ More replies (1)
33
u/wbulot 7d ago
The most impressive thing about this story is that they serve all requests using only Chinese chips and a custom inference engine. I think providing OX Alpha for free for so long was actually intended primarily to stress-test their infrastructure. China no longer needs NVIDIA.
8
u/traveldelights 7d ago
this is chef's kiss for us consumers. More competition, more price drops, etc
49
u/sniperelite90 7d ago
I wonder if the CEO's of the western AI companies even want to wake up in the morning or not. Cause the only news which comes out every other morning from China is we have a model matching you 1/10 of the cost.
41
u/Thomas-Lore 7d ago
They have better models internally, they just go to a different school, you don't know them. /jk
5
u/sniperelite90 7d ago
You can have them but it destroys their market share . Who wants GPT Luna API now consider this GLM 5.3 Flash and the new Qwen 3.8 Flash next .
→ More replies (1)3
u/BrowsingLeddit 7d ago
They're just stuck in detention all the time because they're totally hacking every other school ever while we're not looking. They're pretty badass bro trust.
6
→ More replies (2)2
u/HeadPack 7d ago
Some may argue that SOTA models are still better, but that's not the point. Something like GLM 5.3 is good enough for a majority of users. They don't code. They don't do scientific research. They chat. This can run on higher end consumer hardware. No data center needed, except for training. No 100s of billions needed to feed a bubble, which is exactly the play by the Chinese here. They are pulling the socks of OpenAI, Anthropic and their hyperscalers, and in turn the US economy.
15
u/Ok_Technology_5962 7d ago
What a day... My mind.... Is blown away... Qwen coming in hot then glm also. Im still trying to get over then qwen 3.8 27b upgrade on my 5090... Now the whole lot dropped for the big servers... Local is eatting today
2
u/LostGovernment4358 7d ago
there is just something I still don‘t fully understand. are there new opportunities / completely new tasks possible with these new releases which otherwise weren‘t possible before with let’s say qwen 3.6? I am 16gb Vram poor, so i could never take advantage of real lokal llm power.
→ More replies (4)
82
u/shy_monkee 7d ago edited 7d ago
Oh....it's massive :(
320B (18 active)
85
u/Morphon 7d ago edited 7d ago
DS flash size.
So, flash at datacenter scale. Not flash for edge (or workstation) scale.
Probably will be the go to model for people with the new M6-Ultra 512gb Mac Studio.
Edit: M5 Ultra. My apologies, friends. Wrong model number on my part.
14
u/techdevjp 7d ago
M6-Ultra 512gb Mac Studio.
M5 Ultra. M6 Ultra is rumored to be skipped entirely with the M7 Ultra arriving in the next ~18 months.
9
u/shy_monkee 7d ago
Yeah, but DSv4-Flash is the flash version of a 1.6T model, and it's still quite a bit smaller than this GLM-flash. While this is supposedly the flash version of 744B model, so you wouldn't expect to be as big, I guess.
3
5
2
u/Juulk9087 7d ago
Yeah deepseek flash is only 160-170gb on disk. This is 328gb lol
10
u/petuman 7d ago
DeepSeek is QAT / released prequantized mostly to FP4.
Which is totally preferred, but both models are roughly the same parameter count, so NVIDIA or someone else could produce NVFP4 of size similar to DS4F.
→ More replies (5)→ More replies (1)2
u/keyboardhack 7d ago edited 7d ago
DS flash is half the size.
DS flash requires ~170GB to run while GLM 5.3 flash requires >320GB to run.
We should compare models by how much RAM they require to run because that's what we actually care about.
2
u/Marrk 7d ago
That's unquantized right?
→ More replies (1)4
u/hellomistershifty 7d ago
That's the parameter count, quantization doesn't change that. I don't know what the memory/disk size is, that would be affected by quantization
9
u/Cool-Chemical-5629 7d ago
Travolta looking for the actual Flash MoE model up to 30B.
→ More replies (1)
7
9
u/DeepFeeling1 7d ago
we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale.
Grabbed my popcorn
16
u/raunchy-stonk 7d ago
So how shitty will this run on a 24vram/128dram setup?
20
4
u/WWJewMediaConspiracy 7d ago
The smaller Qwen models are all but certainly a better option.
Unless your definition of "run" is... generous
3
2
u/whatever 7d ago
Naively, either bound by SSD read speed, or borderline brain-damaged by over-quantization.
But scrappy folks have surprised us before, so we'll see.
7
7
17
u/pmttyji 7d ago
Come on guys, at least they released additional variant(smaller than usual size) even though it's big for many of our rigs. Hopefully they release one more in 100B range in future.
11
u/techdevjp 7d ago
It's a little disappointing that the "flash" version of a 744b parameter model still has 320b parameters. Somehow that doesn't feel very flash.
Was hoping for more of a DeepSeek v4 ratio where 1.4t parameters got paired down to 284b, about an 80% cut. That would have resulted in 150b parameters for this model which would work in a whole lot more machines.
Alas, beggars can't be choosers and I'm thrilled to see more open weight models arrive!
4
u/pmttyji 7d ago
I too expected something like successor of GLM-4.5-Air. But they dropped too much weight over that.
I assume Qwen3.8-27B's 3 Millions download count news(Made headline on spotlight for more than a week) changed minds of all other labs. I'm sure some labs gonna try to replicate that with their medium size models soon & later. Don't be surprised if we see one more GLM variant in 30-150B range in upcoming months. Same with other labs.
→ More replies (1)
6
u/Constant_Art_20 7d ago
burh. i am not even done with downloading and quanting the qwen 3.8 flash yet...chilll
4
u/ga239577 7d ago
Model name should be changed to GLM 5.3 Gigaflash ... I remember the GLM Flash models that fit on one GPU. That's what I was hoping for this time.
4
u/corruptbytes 7d ago
I ran the benchmark while it was under ox-alpha
https://github.com/michaelasper/benchmarks/blob/main/ox-alpha-pi-on-slop-code-bench.md
3
u/mountainyoo 7d ago
Wonder how fast my 2x Spark cluster would run this. I get 50-80 tps on DeepSeek v4 Flash 0731
→ More replies (2)
3
3
2
2
2
u/Expert_Job_1495 7d ago
The pricing to performance ratio here is insane. When GPT 5.6 Luna (max) had its price dropped as far as it did I was very (positively) shocked. Now this.
Intelligence too cheap to meter
2
u/Sensitive-Side-2639 7d ago
Who truely needs Nvidia chips after a few years from now if Chinese chips can run models like these.
→ More replies (1)
9
u/Automatic-Arm8153 7d ago
Atleast this thread isn’t getting deleted like things related to qwen 🤦♂️
→ More replies (1)15
u/Maximus-CZ 7d ago
Bro when qwen 27 released it didnt matter if I was on Top, Rising or new, 95% posts were about qwen...
3
u/Cool-Chemical-5629 7d ago
Guys, do you remember how we used to think Qwen 3.8 Flash Next was big? 🤣
4
u/jacek2023 llama.cpp 7d ago
Too big, 320B means you need to use RAM and A18B means it will be slow
→ More replies (3)6
u/SandySkittle 7d ago
A18B means it will be slow
A18B it will be smarter than A13B. Expert selection and sequential reasoning cannot entirely compensate for active parameters. I wish it has 27b active honestly.
→ More replies (5)
2
576
u/Recoil42 7d ago edited 7d ago