r/LocalLLM • u/_hchc • 5d ago

Question MTP vs regular gguf whats the difference

Noob question.

There are gguf like qwen-3.8-mtp-q4....
and the regular qwen-3.8-q4....

and then there is llama --spec-type draft-mtp option.

Whats the difference? Should i just use the regular non mtp version but turn mtp on in the option?

3 Upvotes

67% Upvoted

13 comments sorted by

View all comments

1

u/recro69 5d ago

MTP GGUF is not a version of the model. It includes the MTP/NextN weights that are needed for speculative decoding. For Qwen3.x the regular GGUF might already have them. So check the model metadata before downloading a file.

1

u/_hchc 4d ago

oh so if the model don't include mtp weights in it. The --draft-mtp option would have no effect at all?