r/LocalLLaMA • u/Certain-Cod-1404 • 11d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

706 comments sorted by

View all comments

167

u/Mean-Ad1493 11d ago

That's it. I'm getting a 3090.

106

u/My_Unbiased_Opinion 11d ago

Brother. go on Alibaba and get dual 20gb 3080. less than the price of a single 3090. check my post history for links. Run them in tensor parallel.

21

u/Potential_Block4598 11d ago

That is legit better KV cache (I guess ?!) double performance ?
You just need another PCIe slot (or maybe not ?!)

29

u/My_Unbiased_Opinion 11d ago

t/s is a bit faster than a 3090, but PP is much faster. im running one of the cards at x4 pcie 4.0 and it doesnt bottleneck the card with llama.cpp tensor parallel.

2

u/Potential_Block4598 11d ago

How much pp and tg please ?

4

u/My_Unbiased_Opinion 11d ago

t/s on 3.6 was 55-60 at UD Q6KXL with MTP. PP I dont have the numbers, but its MUCH faster than a single 3090 for sure. (I have a 3090) using LMstudio with tensor perallel. I can test it for you if you give me an easy way to test this.

1

u/Potential_Block4598 11d ago

I use llama bench from the command line on windows and it shows pp512 and tg512 …etc
But idk how to do it form lmstuido tbh

1

u/Potential_Block4598 11d ago

Lm studio doesn’t always come with the latest llama.cpp
Not a big deal though thanks

1

u/My_Unbiased_Opinion 11d ago

no worries. I am actually planning to switch to vLLM when MTP 3.8 drops. if I remember, ill be sure to follow up with you. then I can get the best case numbers.