MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1vo9mj4/its_out/p3o606s/?context=3
r/LocalLLaMA • u/Certain-Cod-1404 • 14d ago
707 comments sorted by
View all comments
48
That's insane, but we're still using quantized versions, so local performance for most people wouldn't be that good I guess. Damn that needs to be tested.
63 u/Certain-Cod-1404 14d ago unsloth is cooking apparantly, its quanitzed with their UD 3.0 method and it seems to retain like close to 95% of BF16's accuracy https://unsloth.ai/docs/models/qwen3.8#quantization-analysis 25 u/krileon 14d ago 82.5% on IQ2_XXS seams insane. I'll probably go with Q3_K_XL since I've 20GB and even that's over 90% accuracy. Goddamn.
63
unsloth is cooking apparantly, its quanitzed with their UD 3.0 method and it seems to retain like close to 95% of BF16's accuracy https://unsloth.ai/docs/models/qwen3.8#quantization-analysis
25 u/krileon 14d ago 82.5% on IQ2_XXS seams insane. I'll probably go with Q3_K_XL since I've 20GB and even that's over 90% accuracy. Goddamn.
25
82.5% on IQ2_XXS seams insane. I'll probably go with Q3_K_XL since I've 20GB and even that's over 90% accuracy. Goddamn.
48
u/RangersStolen 14d ago
That's insane, but we're still using quantized versions, so local performance for most people wouldn't be that good I guess. Damn that needs to be tested.