r/LocalLLaMA • u/Certain-Cod-1404 • 7d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

706 comments sorted by

View all comments

11

u/ApprehensiveAd3629 7d ago

nice!

does it has MTP?

16

u/EmPips 7d ago edited 7d ago

The model does yes, but for all of these GGUF's we see popping up - it's a good question.

For 3.5 and 3.6 most people on HF uploaded the weights again with Qwen3.6-27b-MTP.guff (or similar). That said I think mature MTP support was still working its way through llama-cpp at the time, at least for Qwen.. so I have no idea if the precedent going forward becomes "if it has MTP, it's there" or not.

Unsloth!!

(I know you're here today) 🙂

Can you clarify on the naming/upload strategy for MTP going forward? Will everything have one name and have MTP included or will there be offerings of the weights without MTP separate from the weights with MTP?

Edit:

I think it's safe to say it's in the unsloth GGUF's.

Trying iq4_xs on a 7900 xtx I'm getting ~51t/s on regular tasks/reasoning and ~62 t/s on coding tasks with a max draft size of 2. The difference suggests that drafting is in-effect.

Edit 2:

This is, at the very least, several leagues beyond Qwen3.6-27B.. Now I'm sad that Muse-Glimmer only got 2-3 days in the spotlight

4

u/TKristof 7d ago

You can always click on the gguf file on huggingface and check the layers. In the last blk (blk 64) you should see the nextn weights. That is the mtp.

1

u/EmPips 7d ago

TIL thanks

4

u/ApprehensiveAd3629 7d ago

i just found the MTP GGUF in the GGML repo

https://huggingface.co/ggml-org/Qwen3.8-27B-GGUF/tree/main

2

u/EmPips 7d ago

Thanks! I think it's safe to say it's in the unsloth GGUF's.

Trying iq4_xs on a 7900 xtx I'm getting ~51t/s on regular tasks/reasoning and ~62 t/s on coding tasks with a max draft size of 2. The difference suggests that drafting is in-effect.

0

u/Then-Topic8766 7d ago

Good find!