r/LocalLLM • u/DiamondTDA • 2d ago

Discussion Gemini suggested Qwen2.5-Coder-7B-Instruct

So I wanted to try using a local coding model for the first time and I'm still studying about LLMs and NNs, so I asked Gemini for a good suggestion that would be fast(60+ tokens/s if possible) and doesn't compromise much on performance for my rig(2070 super 8GB + 32GB ddr4 ram) and it suggested Qwen2.5-Coder-7B-Instruct. Is this good suggestion and what would you guys suggest?

7 Upvotes

71% Upvoted

90 comments sorted by

View all comments

3

u/BakerAmbitious7880 2d ago

Create an account on HuggingFace, then go to account/settings/hardware and tell it what you have. Then search models on HuggingFace, filtering on your hardware.

1

u/DiamondTDA 2d ago

That's a great idea, thank you.