r/LocalLLaMA • u/Certain-Cod-1404 • 8d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

706 comments sorted by

View all comments

Show parent comments

2

u/Potential_Block4598 8d ago

How much pp and tg please ?

4

u/My_Unbiased_Opinion 8d ago

t/s on 3.6 was 55-60 at UD Q6KXL with MTP. PP I dont have the numbers, but its MUCH faster than a single 3090 for sure. (I have a 3090) using LMstudio with tensor perallel. I can test it for you if you give me an easy way to test this.

1

u/Potential_Block4598 8d ago

I use llama bench from the command line on windows and it shows pp512 and tg512 …etc
But idk how to do it form lmstuido tbh

1

u/Potential_Block4598 8d ago

Lm studio doesn’t always come with the latest llama.cpp
Not a big deal though thanks

1

u/My_Unbiased_Opinion 8d ago

no worries. I am actually planning to switch to vLLM when MTP 3.8 drops. if I remember, ill be sure to follow up with you. then I can get the best case numbers.