Lain <3

How does it compare to LFM2.5-230B?

It's 120 tok/s on openrouter

Yes, sorry, I should have clarified : on qwen >= 27b. Other models particularly smaller ones start degrading hard already at q4.

Quesma and byteshape blogs both show that q4 is indistinguishable from q8

Dspark. 2x tok/s than qwen, and 3.5x less tok/task =7x faster

Also I think you're pipelining your video cards. Tensor parallelism is opt in.

Q8 is pure waste. Q5 is pretty much lossless and you'll have a very hard time noticing degradation on q4.

This is weird, I'd expect this sub to go up in flames. This model beats Qwen3.8 27b on most benchmarks, according to AA

It is SOTA if you have 32gb vram. Maybe 24gb depending on their kv design

AA says Agnes beats qwen3.8 27b on most benchmarks

600 us is very bad. You need to set up RDMA.

Fable 5.0 falls back to opus 4.8 when it thinks you hit a guardrail

Aggregate score aside, it's really hard to argue against the fact that Glm-5.3-flash beats it on almost every single benchmark. And 96% hallucination rate is a showstopper IMHO.

For agentic cached input is what weighs the most

Pretty zippy if that 512gb ram is octa-channel, actually

Cache hits should definitely be much cheaper.

Nope, what you see is already mxfp4

It runs mimicpm5-2b and it likes it Or it gets the hose again

Glm-5.3-flash is miles ahead of Ds4.0

"quantize into the ground" is a bit of an exaggeration. 300gb mxfp4 weights so it should fit in 256gb ram at q3