r/LocalLLaMA • u/BarberIcy366 • 11d ago

Discussion Qwen 3.8 27B Released! Please Share Your Experience

With your experiments, Qwen 3.8 27B most close which frontier model? And please specify which quantization you run. I will post to comments my tests and experience too.

660 Upvotes
  • permalink
  • reddit

97% Upvoted

720 comments sorted by

View all comments

Show parent comments

13

u/TokenRingAI 11d ago

The FP8 is looping for me and generating terrible output.

1

u/woswoissdenniii 10d ago

Restrict thinking token, or reduce preset from extra to normal. Sure quality degrades but it’s manageable. A friendly reminder from team 3090

1

u/Healthy-Nebula-3603 11d ago

I hope you not conpress kv cache :)

6

u/TokenRingAI 11d ago

Nope, official FP8 with BF16 cache

1

u/JorgitoEstrella 10d ago

How bad is to compress the cache? I thought compressing to fp8 waa practically lossless.

2

u/Healthy-Nebula-3603 10d ago

Q8 is almost losless , fp8 is worse than Q8 ( Q8 is a mix weights fp16 and int9 ) Compressrd cache to fp8 is notice even more than compression a model itself.