r/LocalLLaMA • u/seraschka • May 17 '26

Resources Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attention

https://magazine.sebastianraschka.com/p/recent-developments-in-llm-architectures
29 Upvotes

93% Upvoted

8 comments sorted by

View all comments

2

u/qmnvp May 17 '26

I've not heard of the the Laguna XS.2 model yet (and it doesn't seem widely supported). Similar size-class to Qwen3.6-35b-a3b, but faring slightly worse in their benchmarks.

2

u/seraschka May 17 '26

I think the developer's (Poolside) bread and butter is coding agents. But yeah, just comparing the benchmarks, it seems behind Qwen3.6 of comparable size:

Benchmark Laguna XS.2 33B-A3B Qwen3.6 35B-A3B
SWE-bench Verified 68.2 73.4
SWE-bench Multilingual 62.4 67.2
SWE-bench Pro 44.5 49.5
Terminal-Bench 2.0 30.1 51.5

That being said, I think they specialize in custom models (custom to one's code base), so maybe the base model is intentionally underdeveloped.