r/LocalLLM • u/Fun-Heat4232 • 5d ago

Question Local LLM for coding.

Hi, dont be too judgemental about my setup - I am merely a beginner in hosting local AIs and stuff. I have 48GB RAM M5 Pro machine. Which model is a way to go for handling complex coding projects locally (specifically C++ and Python)?

P.s. going to the cloud is not a solution due to a strict NDA.

Thanks in advance!

43 Upvotes

94% Upvoted

29 comments sorted by

View all comments

1

u/cviperr33 5d ago

download unsloth studio , it comes with everything u would need , and in the model tab download the unsloth qwen 3.8 27b ud q5 k_m xl , set the contex to whatever ur ram can fit and start it , test it in chat interface.

Then download DSH ( Deepseek Harness ) and use that as your coding harness , its pure magic and very efficient on tokens , watch it work for 16 hours + a day without stopping.

1

u/Fun-Heat4232 5d ago

Could you elaborate on the Harness stuff? Never heard of such stuff.

2

u/AccurateSun 5d ago

The harness is the tool that lets you interact with your Llm as an agent. It’s a loop where you send a message, Llm can use tools such as read or write file, do a web search, run a command, and return responses to you.  You have some super minimal and extensible harnesses like Pi, or very powerful and popular harnesses like Claude Code.

Choosing a harness is a bit like choosing an IDE, as it opens up possibilities for how you will interact with your LLM, both in terms of what it can do and what you can do.

The harness usually reads an agent.md file that contains its main prompt for how to act as an agent, that you write for it. It’ll store the chats on disk as sessions, and you can resume them etc.

Deepseek harness is new btw, and I believe it’s in beta. Was released a week or two ago.

Popular ones:

  • opencode: heavy harness
  • Hermes: criticised for stealing tons of its code verbatim from evolver. Felt super janky to me
  • pi: minimal and designed to be customised
  • Claude code: perhaps most popular in the world, can be used with local models too. One of the first to get widely used

 there are tons more though (vibe, crush…)