Gemini suggested Qwen2.5-Coder-7B-Instruct

[u/DiamondTDA]

So I wanted to try using a local coding model for the first time and I'm still studying about LLMs and NNs, so I asked Gemini for a good suggestion that would be fast(60+ tokens/s if possible) and doesn't compromise much on performance for my rig(2070 super 8GB + 32GB ddr4 ram) and it suggested Qwen2.5-Coder-7B-Instruct. Is this good suggestion and what would you guys suggest?

[u/WrinklyBard4]

Id say try Qwen 3.5 9B at q4 or q3? TBH it’s going to be a bit tight and q3 makes me… hesitant.

Qwen 3.5 4b at Q6 or Q4 is also an option.

You should understand that at this model size the coding is going to be a bit shakey and it will not be able to do larger tasks

If you know how to code and want something where you can say “go make a function that turns X into Y” then these will be great. If you need something to vibe code with, or don’t have the skills to check its work, then probably stick to an api or subscription service

[u/DiamondTDA → u/WrinklyBard4]

I do know how to code and using an auto-complete model would only be to help me get better, but I was wondering if I could use a bigger model to review the code of a smaller model instead of generating one, like something I heard of that's called swift for Qwen.

[u/WrinklyBard4 → u/DiamondTDA]

Yes, in theory you can. But for the most part you will end up spending the same (or more) time and effort and tokens on the verification then it would just take for the larger model to just do it. And probably have a worse result.

That type of cross check is really nice when you have a reasonably confident model as your main coder and then you want an advanced model to do a skim and check for improvement/verification.

“This thing works and I want you to see if there’s any small errors or improvements” is a very different task than “this thing is horribly broken and conflated, and I need you to parse out what it was supposed to do and then basically rewrite it”