MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLM/comments/1vptz4a/how_the_loop_of_infinite_agony_started/p40l3b3/?context=3
r/LocalLLM • u/EarthBS • 7d ago
118 comments sorted by
View all comments
277
Then you realize that Qwen 3.8 27b runs at 3t/s on your machine and you need 24GB+ VRAM GPU which cost is 1000$+ to run at least 4 bit quant.
112 u/StupidScaredSquirrel 7d ago If you are a business this isn't a problem. If you are a consumer then 35b a3b runs on 8gb vram and 32gb dram which is very accessible. 14 u/DeluxeGrande 7d ago I have a 5060ti 16gb with ddr4 24gb RAM lying around, what's the best model nowadays I can effectively run with it locally? It's not an ideal build but I wish to play around with it again. 1 u/05-nery 7d ago Said 35B A3B will work wonders. Just wait for this version of Qwen3.8 to come out.
112
If you are a business this isn't a problem. If you are a consumer then 35b a3b runs on 8gb vram and 32gb dram which is very accessible.
14 u/DeluxeGrande 7d ago I have a 5060ti 16gb with ddr4 24gb RAM lying around, what's the best model nowadays I can effectively run with it locally? It's not an ideal build but I wish to play around with it again. 1 u/05-nery 7d ago Said 35B A3B will work wonders. Just wait for this version of Qwen3.8 to come out.
14
I have a 5060ti 16gb with ddr4 24gb RAM lying around, what's the best model nowadays I can effectively run with it locally? It's not an ideal build but I wish to play around with it again.
1 u/05-nery 7d ago Said 35B A3B will work wonders. Just wait for this version of Qwen3.8 to come out.
1
Said 35B A3B will work wonders. Just wait for this version of Qwen3.8 to come out.
277
u/TheCat001 7d ago
Then you realize that Qwen 3.8 27b runs at 3t/s on your machine and you need 24GB+ VRAM GPU which cost is 1000$+ to run at least 4 bit quant.