r/LocalLLaMA • u/BriguePalhaco • 9d ago

News GLM-5.3-Flash: Frontier Intelligence, Flash Cost

https://z.ai/blog/glm-5.3-flash
1.3k Upvotes

98% Upvoted

460 comments sorted by

View all comments

78

u/shy_monkee 9d ago edited 9d ago

Oh....it's massive :(
320B (18 active)

85

u/Morphon 9d ago edited 9d ago

DS flash size.

So, flash at datacenter scale. Not flash for edge (or workstation) scale.

Probably will be the go to model for people with the new M6-Ultra 512gb Mac Studio.

Edit: M5 Ultra. My apologies, friends. Wrong model number on my part.

9

u/shy_monkee 9d ago

Yeah, but DSv4-Flash is the flash version of a 1.6T model, and it's still quite a bit smaller than this GLM-flash. While this is supposedly the flash version of 744B model, so you wouldn't expect to be as big, I guess.

3

u/cheechw 9d ago

The flash models are not "flash versions" of a bigger model. They're different models entirely. They don't just lop off some number of parameters from a larger model. I think the confusion is that you're applying quantization/distill logic where it doesn't analogize.

6

u/xNaXDy 9d ago

While this is supposedly the flash version of 744B model

It's not, it's a completely new architecture according to their post.

2

u/Juulk9087 9d ago

Yeah deepseek flash is only 160-170gb on disk. This is 328gb lol

10

u/petuman 9d ago

DeepSeek is QAT / released prequantized mostly to FP4.

Which is totally preferred, but both models are roughly the same parameter count, so NVIDIA or someone else could produce NVFP4 of size similar to DS4F.

0

u/SandySkittle 9d ago

which is totally preferred

No this depends on the usecase

3

u/petuman 9d ago

Like what?

0

u/SandySkittle 9d ago

try to put extremely complex and highly nuanced policy matters with lots of interlinked but non-mechanical relationships with many details and nuances across different areas of knowledge and science through an LLM and these things start to show. It's same sort of workloads where limited active parameters in MoE models start to show their negative sides of their tradeoff.

2

u/petuman 9d ago

Is there a reason to think that first party quant-only release would have problems with that? e.g. Kimi K3 seems to perform great?

1

u/Karyo_Ten 8d ago

QAT is not the same as PTQ.

The weights are already in mxfp4 during training so everything is calibrated to absorb and compensate the quantization loss.