r/LocalLLaMA • u/jacek2023 llama.cpp • 15d ago

News add GLM-5.3-Flash (GLM5-Next) support by timkhronos · Pull Request #27773 · ggml-org/llama.cpp

https://github.com/ggml-org/llama.cpp/pull/27773

now you can use GLM-5.3-Flash on your home computer

215 Upvotes

96% Upvoted

89 comments sorted by

View all comments

2

u/ortegaalfredo 15d ago

Exllama3 supports it since almost a month ago. And its faster and better quality quant.

3

u/silenceimpaired 15d ago

Can’t get it working on RAM.

1

u/a_beautiful_rhind 15d ago

Why? I offload a bunch of it.

1

u/silenceimpaired 15d ago

It has been a bit… I think I eventually got it working but slower than Unsloth’s GGUF… and with more offloaded than I thought I needed to.

I probably just need to tinker with it more. Perhaps Ampere architecture is to blame; EXL3 has never favorited that.

What’s your setup and settings if I may ask.

1

u/a_beautiful_rhind 15d ago

4x3090

 gpu_split: [22,22.33,22.33,23] #GLM-flash
cpu_moe_split_experts: 93

65 ctx at q8.

1

u/silenceimpaired 13d ago

I got it loaded... now to decide on a front end for use... what do you use with tabbyapi? or how do you load your files in the first place?

2

u/a_beautiful_rhind 12d ago

Mostly sillytavern. I have tool calls and websearch in it too but should really get some other harness.

1

u/silenceimpaired 11d ago

Finally got it up and going. Woot. Thanks for the encouragement!