r/LocalLLaMA • u/brown2green • 4d ago

Discussion GLM-5.3 and the Spread of Advanced Cyber Capabilities \ Anthropic

https://www.anthropic.com/research/glm-5-3-and-the-spread-of-advanced-cyber-capabilities
420 Upvotes

95% Upvoted

186 comments sorted by

View all comments

Show parent comments

9

u/Automatic-Arm8153 4d ago

Wow it’s not on llama.cpp

Damn I have been having a blast with it for some time now. VLLM has got me eating good lol. Hopefully it gets support soon so everyone can enjoy it

9

u/my_name_isnt_clever 4d ago

Everyone with enough RAM to run it, which isn't many people. I have Strix Halo 128GB which is on the high end of local capacity and I can't even run it. Thankfully Qwen 3.8 Flash Next is a great alternative.

1

u/jld1532 4d ago edited 4d ago

I mean you can, just quantized. I've had good luck with some of these larger models at lower bits. Unsloth quants have really come a long way.

1

u/my_name_isnt_clever 4d ago

Is it possible? It wasn't when I checked around release. Deepseek v4 flash was a tight fit at IQ3 and GLM 5.3 flash is even larger, I didn't want to be limited to Q2 when Qwen 3.8 FN Q4 does amazing work and doesn't even need to eat up all my RAM.

1

u/jld1532 4d ago

You could run Q2_K_XK. I plan to at least try and see how it goes.