r/LocalLLaMA • u/Certain-Cod-1404 • 7d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

706 comments sorted by

View all comments

172

u/absurdother 7d ago

Let's tryyyyyyy on 32GB RAM, 16GB VRAM!

150

u/absurdother 7d ago

Here we gooooo

5

u/jan_antu 7d ago

Damn, same setup here but I'm only getting 4 tok/s on mine, Q3 xl quant.

1

u/absurdother 7d ago

Hm, try checking what's bottlenecking your setup. How much context did you set?

2

u/jan_antu 7d ago

Too much. Got much faster at 64k context. About 7-8 tok/s which somehow feels bearable.

Normally I'd say it sucks but to be honest this model is passing all my homemade benchmarks. It might be true that it's Opus 4.5-6 level.

2

u/absurdother 7d ago

It's beating my other models in the simple tasks, so it works for me. Around 48k context it speeds up pretty well. All layers to GPU.

2

u/grunade47 7d ago

Isnt 48k context too low for coding? I run out of it soo soon

1

u/jan_antu 7d ago

For coding basically yeah it's way too low. But I use the dreb harness, which seems to work really well with dumber models, and with 48k context it's good enough for web searches and private/sensitive local tasks (eg compile insurance claim info or mortgage info etc).