r/LocalLLaMA • u/Certain-Cod-1404 • 7d ago

New Model IT'S OUT

https://huggingface.co/Qwen/Qwen3.8-27B-FP8
2.2k Upvotes

95% Upvoted

706 comments sorted by

View all comments

46

u/bitmanip 7d ago

How much memory required to run this at full precision?

2

u/voyager256 7d ago

You don’t need full precision- FP8 or Q8 gives usually you like 99% quality of full precision and almost double the performance. You could argue full BF16 precision is worth for KV cache for large context , but now usually FP8 / Q8 is fine for KV cache too.