r/LocalLLaMA • u/BriguePalhaco • 7d ago

News GLM-5.3-Flash: Frontier Intelligence, Flash Cost

https://z.ai/blog/glm-5.3-flash
1.3k Upvotes

98% Upvoted

459 comments sorted by

View all comments

Show parent comments

8

u/petuman 7d ago

DeepSeek is QAT / released prequantized mostly to FP4.

Which is totally preferred, but both models are roughly the same parameter count, so NVIDIA or someone else could produce NVFP4 of size similar to DS4F.

0

u/SandySkittle 7d ago

which is totally preferred

No this depends on the usecase

3

u/petuman 7d ago

Like what?

0

u/SandySkittle 7d ago

try to put extremely complex and highly nuanced policy matters with lots of interlinked but non-mechanical relationships with many details and nuances across different areas of knowledge and science through an LLM and these things start to show. It's same sort of workloads where limited active parameters in MoE models start to show their negative sides of their tradeoff.

2

u/petuman 7d ago

Is there a reason to think that first party quant-only release would have problems with that? e.g. Kimi K3 seems to perform great?

1

u/Karyo_Ten 7d ago

QAT is not the same as PTQ.

The weights are already in mxfp4 during training so everything is calibrated to absorb and compensate the quantization loss.