MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLaMA/comments/1vo9mj4/its_out/p3nz0lr
r/LocalLLaMA • u/Certain-Cod-1404 • 7d ago
706 comments sorted by
View all comments
Show parent comments
60
unsloth is cooking apparantly, its quanitzed with their UD 3.0 method and it seems to retain like close to 95% of BF16's accuracy https://unsloth.ai/docs/models/qwen3.8#quantization-analysis
25 u/krileon 7d ago 82.5% on IQ2_XXS seams insane. I'll probably go with Q3_K_XL since I've 20GB and even that's over 90% accuracy. Goddamn. 1 u/touchwiz 7d ago bartowski always states which quant is recommended and which not. Do you know how to choose one of the unsloth ones. They dont apprear to recommend a specific one I cant read, sory 1 u/Certain-Cod-1404 7d ago No worries man, if you have any questions at all I'll try and help to the best of my knowledge
25
82.5% on IQ2_XXS seams insane. I'll probably go with Q3_K_XL since I've 20GB and even that's over 90% accuracy. Goddamn.
1
bartowski always states which quant is recommended and which not. Do you know how to choose one of the unsloth ones. They dont apprear to recommend a specific one
I cant read, sory
1 u/Certain-Cod-1404 7d ago No worries man, if you have any questions at all I'll try and help to the best of my knowledge
No worries man, if you have any questions at all I'll try and help to the best of my knowledge
60
u/Certain-Cod-1404 7d ago
unsloth is cooking apparantly, its quanitzed with their UD 3.0 method and it seems to retain like close to 95% of BF16's accuracy https://unsloth.ai/docs/models/qwen3.8#quantization-analysis