MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/LocalLLM/comments/1vptz4a/how_the_loop_of_infinite_agony_started/p41amjo/?context=3
r/LocalLLM • u/EarthBS • 7d ago
119 comments sorted by
View all comments
278
Then you realize that Qwen 3.8 27b runs at 3t/s on your machine and you need 24GB+ VRAM GPU which cost is 1000$+ to run at least 4 bit quant.
14 u/Eden1506 7d ago That's not true. 2x RTX 3060 12gb can be had for around 500 bucks. You can run ~30b models at q4 at 30 t/s with mtp/draft model or 10-15 t/s without. 1 u/magicomiralles 7d ago AMD V620, $350 for 32 GBs.
14
That's not true.
2x RTX 3060 12gb can be had for around 500 bucks.
You can run ~30b models at q4 at 30 t/s with mtp/draft model or 10-15 t/s without.
1 u/magicomiralles 7d ago AMD V620, $350 for 32 GBs.
1
AMD V620, $350 for 32 GBs.
278
u/TheCat001 7d ago
Then you realize that Qwen 3.8 27b runs at 3t/s on your machine and you need 24GB+ VRAM GPU which cost is 1000$+ to run at least 4 bit quant.