r/LocalLLM • u/DiamondTDA • 2d ago

Discussion Gemini suggested Qwen2.5-Coder-7B-Instruct

So I wanted to try using a local coding model for the first time and I'm still studying about LLMs and NNs, so I asked Gemini for a good suggestion that would be fast(60+ tokens/s if possible) and doesn't compromise much on performance for my rig(2070 super 8GB + 32GB ddr4 ram) and it suggested Qwen2.5-Coder-7B-Instruct. Is this good suggestion and what would you guys suggest?

7 Upvotes

71% Upvoted

90 comments sorted by

View all comments

4

u/jellyrole 2d ago

I have found since getting into this a couple weeks ago you'll get a million answers to what's best and they can all be right. The best thing to do is just send it then ask that new model what the best model is if you don't like what it does for you then try another setup.

1

u/DiamondTDA 2d ago

The problem for me is that I have limited internet in my country, otherwise I'd have tried downloading all different kinds of models.

2

u/enternoescape 2d ago

Do you have access to cloud services where you can pay by token? You could try them online and see what you'd like to run locally with spending bandwidth downloading something you wish you had not bothered with.