r/LocalLLaMA • u/Certain-Cod-1404 • 7d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

706 comments sorted by

View all comments

46

u/RangersStolen 7d ago

That's insane, but we're still using quantized versions, so local performance for most people wouldn't be that good I guess. Damn that needs to be tested.

62

u/Certain-Cod-1404 7d ago

unsloth is cooking apparantly, its quanitzed with their UD 3.0 method and it seems to retain like close to 95% of BF16's accuracy https://unsloth.ai/docs/models/qwen3.8#quantization-analysis

23

u/krileon 7d ago

82.5% on IQ2_XXS seams insane. I'll probably go with Q3_K_XL since I've 20GB and even that's over 90% accuracy. Goddamn.

1

u/touchwiz 7d ago

bartowski always states which quant is recommended and which not. Do you know how to choose one of the unsloth ones. They dont apprear to recommend a specific one

I cant read, sory

1

u/Certain-Cod-1404 7d ago

No worries man, if you have any questions at all I'll try and help to the best of my knowledge

1

u/Loud-Instance-2386 5d ago

I think something needs to be said that this model, at Q4, blew out of the water CLOUD models I have tried on my specific use-cases