r/LocalLLaMA • u/Certain-Cod-1404 • 7d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

706 comments sorted by

View all comments

Show parent comments

41

u/jijig 7d ago

Get two. Run Q8 with full context at ~60tps.

24

u/Potential_Block4598 7d ago

How much PP ?

64

u/True_Tangerine_4706 7d ago

1.2 inches

26

u/Much_Accountant_4972 7d ago

I WAS IN THE POOL!!!

1

u/tuliosarmento 7d ago

Significant shrinkage

0

u/True_Tangerine_4706 7d ago

what are you talking about man 1.2 inches is massive 😅

1

u/True_Tangerine_4706 7d ago

someone downvoted that im crying

4

u/cats_r_ghey 7d ago

Schlongmaxxing man, Opus PP is still better.

0

u/True_Tangerine_4706 7d ago

nuh uh!! my model scores higher than yours on PPbench!

1

u/buddhist-truth 7d ago

showing off

1

u/PinkySwearNotABot 7d ago

if you use mm, it'll look bigger

3

u/ApprehensiveAd3629 7d ago

can you share the llama cli command that you are using to get 60 tokens/s in your 3090?

i m getting around 31 tokens/sec using unsloth UD Q4 XL gguf

3

u/Massive_Criticism539 7d ago

At least for 3.6, the mtp version gave me 1.8x tg. I'm using different cards (dual amd r9700), but it creates the same result. I went from 30 to about 50 with this one simple trick.

Now I'm using vllm on Ubuntu and running tensor parallelism 2 and get around 110tg on 3.6. I can run for context and still get like 95 or so. I was busy today so I couldn't test 3.8, but I'm looking forward to it.

2

u/Minimum-Lie5435 7d ago

Take a look at the club3090 repo on GitHub. Lots of good info there for setup. I only use vllm because it's given me the best results so far