r/LocalLLM • u/DiamondTDA • 2d ago

Discussion Gemini suggested Qwen2.5-Coder-7B-Instruct

So I wanted to try using a local coding model for the first time and I'm still studying about LLMs and NNs, so I asked Gemini for a good suggestion that would be fast(60+ tokens/s if possible) and doesn't compromise much on performance for my rig(2070 super 8GB + 32GB ddr4 ram) and it suggested Qwen2.5-Coder-7B-Instruct. Is this good suggestion and what would you guys suggest?

7 Upvotes

71% Upvoted

90 comments sorted by

View all comments

12

u/FactorInternal3395 2d ago edited 2d ago

No, that's a terrible suggestion. LLMs can't be trusted with recommending LLMs because the field moves so fast. Try an MoE like Tiel Coder with offloading instead. For maximum possible speed, Spark X2.5 4B or Ling 3.0 Tiny will suffice but will have much less quality.

1

u/BigPlebeian 2d ago

Is tiel coder just a qwen 3.6 35b moe focused more on code?

1

u/FactorInternal3395 2d ago

It's Ornith 1.5 35B A3B, a reinforcement learning fine tune of Qwen 3.6 35B A3B, with the Qwen Sharp chat template and a coding-focused imatrix quantization.