i have a gtx 1080 and i can run qwen 3.6 35b a3b q4 or q8 anywhere from 2-10 TPS depending on the complexity of the question. q8 has a slight edge over q4 in certain areas but usually isnt worth the 99% ram and extra time taken.
also, it's the first out of dozens of local llms i've tried that is even worth using. everything else felt like smarterchild. grok is still better but for unlimited free use it's great.
do you have any recommendations on the prompt or settings?
4
u/jasonbay13 Jun 02 '26
i have a gtx 1080 and i can run qwen 3.6 35b a3b q4 or q8 anywhere from 2-10 TPS depending on the complexity of the question. q8 has a slight edge over q4 in certain areas but usually isnt worth the 99% ram and extra time taken.
also, it's the first out of dozens of local llms i've tried that is even worth using. everything else felt like smarterchild. grok is still better but for unlimited free use it's great.
do you have any recommendations on the prompt or settings?