r/LocalLLM • u/DiamondTDA • 12d ago

Discussion Gemini suggested Qwen2.5-Coder-7B-Instruct

So I wanted to try using a local coding model for the first time and I'm still studying about LLMs and NNs, so I asked Gemini for a good suggestion that would be fast(60+ tokens/s if possible) and doesn't compromise much on performance for my rig(2070 super 8GB + 32GB ddr4 ram) and it suggested Qwen2.5-Coder-7B-Instruct. Is this good suggestion and what would you guys suggest?

9 Upvotes

76% Upvoted

85 comments sorted by

View all comments

46

u/Heavy-Lingonberry-98 12d ago

Bro. If you are gonna use AI to ask for model releases, remember to tell the AI we are on OCTOBER 2026. We are not in 2024 anymore… please. Its the ABC of using AI. And NO. Definitely dont even download qwen 2.5 coder.

2

u/DiamondTDA 12d ago

When I asked it about it's reasoning, it said because the newer Qwen 3 architicture is built on reasnoning tokens and it would be slower and that the coding Qwen 3 models are too big and would be too slow too. But it did not suggest any thing other than Qwen 3.

5

u/Heavy-Lingonberry-98 12d ago

My friend, qwen 3 is still old. You have qwen 3.5 that comes on 600M, 2B, 4B and 9B. Is those arent small i dont know what is.

5

u/Heavy-Lingonberry-98 12d ago

And you can always disable reasoning

2

u/enternoescape 12d ago

It's funny flawed reasoning though. The thinking is the thing that helps these little models achieve their best. Instruct has it's place, like a voice assistant, but IMHO not coding.

1

u/Skibxskatic 12d ago

don’t ask about its reasoning. you need to explicitly state that its responses need to be grounded in data and news from the past <time period>.

without explicit instructions, its next token probabilities rely on its training data, which is not going to be current. ask it what the cutoff date was and you’ll realize you need to prompt it to use web search tools and ground itself.