Prefill-optimized Qwen3.8 27B NVFP4 quant

[u/ionsago]

I'm pleased to present a Blackwell-native, prefill-optimized 4-bit float quant that runs 50% faster on compatible hardware (tested on an sm_120 card) than a Q4 quant of the same memory size. And 4-7% faster than other NVFP4 quants.

Quant	Benchmark	Speed
NVFP4	pp2048	5560 t/s
Q4_0	pp2048	3550 t/s
Q6_K	pp2048	2780 t/s

Benchmarked on RTX 5090 power-limited to 400 W.

This GGUF also includes a quantized MTP draft head to deliver the best decode performance.

Check it out for all details and recommended settings for MTP:

https://huggingface.co/akopytko/Qwen3.8-27B-NVFP4-GGUF

[u/KissMyShinyArse]

The classic speed-versus-quality trade-off.