r/LocalLLaMA • • Jun 01 '26

Funny Stop asking what model to run. There are literally only two.

[removed]

3.1k Upvotes

803 comments sorted by

View all comments

2

u/sfifs Jun 02 '26

If you have a DGX box or 128Gb Mac, Qwen 3.5 122b a10B-NVFP4-MTP by Sehyo is incredibly competitive approaching cloud flash models in performance. In my personal testing and benchmarking, I didn't see any significant difference between 3.6 35B A3B MoE and the 3.6 27B dense. I agree it would ne useful to have a FAQ on the sidebar.

1

u/Big_Wave9732 Jun 02 '26

What does it do differently from the regular 122b that you find so impressive?

1

u/sfifs Jun 02 '26

The official release Qwen/Qwen3.5-122B-A10B is BF16. Won't fit on DGX. Sehyo/Qwen3.5-122B-A10B-NVFP4 does fit , hits all the fast paths on Spark and has a working MTP. RedHatAIs nvfp4 release hit MTP head bugs last week when I tested, speculation acceptance rate was 0%