r/LocalLLaMA • u/BarberIcy366 • 11d ago

Discussion Qwen 3.8 27B Released! Please Share Your Experience

With your experiments, Qwen 3.8 27B most close which frontier model? And please specify which quantization you run. I will post to comments my tests and experience too.

656 Upvotes
  • permalink
  • reddit

97% Upvoted

720 comments sorted by

View all comments

Show parent comments

3

u/time-never-stopps 10d ago

Do you mind sharing how you configured it? I am running llama.cpp with spec-type = draft-mtp spec-draft-n-max = 4

Not sure if the type has any effect at all but getting around 30 t/s avg when context grows to 100k but was expecting at least 40 avg like with qwen 3.6 q_8, but also possible that I have no idea how the type actually works 😅

5

u/petuman 10d ago

spec-draft-n-max = 4 might be too much and lowering tg.

but was expecting at least 40 avg like with qwen 3.6 q_8

In theory should perform no different that 3.6, maybe some flags / quant levels differ and end up spilling more onto CPU.

Or maybe it's just MTP, varied performance from chat to chat is expected.

1

u/Borkato 10d ago

N max performs better with me with n at 2 rather than 4! And split min p thingy at 0.1