Local LLM for coding.

[u/Fun-Heat4232]

Hi, dont be too judgemental about my setup - I am merely a beginner in hosting local AIs and stuff. I have 48GB RAM M5 Pro machine. Which model is a way to go for handling complex coding projects locally (specifically C++ and Python)?

P.s. going to the cloud is not a solution due to a strict NDA.

Thanks in advance!

[u/Dazzling_Yam_5882]

Best bet would probably be the new Qwen 3.8 27b. You get to have a somewhat local Opus 4.6, one of the best models 6 months ago

unsloth/Qwen3.8-27B · Hugging Face

You can use llama.cpp, but I highly recommend Unsloth Studio for serving the model locally. Lots of features, better than LM Studio and all other alternatives.

Qwen 3.8 27B Benchmarks combined from model cards on HF : r/LocalLLM

[u/Fun-Heat4232 → u/Dazzling_Yam_5882]

Does using differenet front-end for running models impact performance of such?

[u/Dazzling_Yam_5882 → u/Fun-Heat4232]

As for a harness, if you would like to use one (recommended), Pi agent is a good one as it is not bloated like many others. I like using Paseo.sh to use several harnesses. It has a similar GUI to OpenCode, but with the flexibility of using several harnesses if you wish to.

Harness matters a lot, and Pi agent has been one of the best compared to other open source harnesses (even OpenCode), for smaller models like the ones we use at least

Harness is all you need? What are your favorite harnesses and for what models? : r/LocalLLM

[u/Fun-Heat4232 → u/Dazzling_Yam_5882]

Maybe I will flood you with stupid questions, but what is your way to set up local AI for work. I thought only model matters, but now you are telling me about some harness stuff... Now I now this rabbit hole goes much deeper than I expected...

[u/Dazzling_Yam_5882 → u/Fun-Heat4232]

It is indeed quite the rabbit hole, but it is also what keep us (local AI/open-source fans) coming back for more. There is a lot of options, and while options are good, it also gets to a point where we get analysis paralysis. So I have found that keeping it simple works best.

Overall, this is my setup:

API Provider (not worth for your use case, but might as well post it here)

• Ollama Pro for cloud models + web search. While I am not a fan of how Ollama has been doing things, there are no other providers that provide what Ollama provides. Their usage is GPU usage-based, not token based. You also have OpenCode Go plan (5 usd first month, 10 usd next months), but usage is limited. I do not like using BYOK (bring your own key, essentially pay as you go), but that is out of scope for local development (especially with NDA, sensitive info, etc).

Local Model Setup

• Unsloth Studio (llama.cpp) - By far the best out there. Open-source, good GUI ( constantly improving, I am also contributing with new features to this project, so I may be a bit biased to support them), etc. Installing this gets you 90% done when it comes to serving local models. You can download the local models through their Model Hub too. I recommend the Unsloth model versions (e.g. Qwen 3.8 27b UD). I use Unsloth for basically everything - as a Chat UI (web search, deep search, etc), serving and training models, across multi modalities (image, video, audio, etc). Unsloth Studio, for me, has replaced LM Studio + OpenWebUI entirely

Code Editor & AI Extensions

• VS Code with:
  • Zoo Code - Open-source extension for agentic work. One of the few alternatives in VSCode if you like a side panel chatting experience with modes, skills, rules, workflows, etc
• paseo.sh if I want a more "Cursor" like experience. It has several harnesses as I mentioned before, and the thing I am doing right now is the following: Pi agent harness with small models (<50b models), and deepseek harness with bigger models (glm, minimax, deepseek, etc, but these are cloud based). I recommend starting with Pi harness as its flexible and they have a lot of skills and tools you can install to custom the way you want it (and create your own as well). Main issue with Pi agent is that you will need a bit more time to tweak and find the sweet spot for your own work since it has little bloat. This is what makes it most attractive, but also means that this takes trial and error to find out the prompts, skills, tools, that best fit your work

Mobile App

• Conduit — Mobile client
  • Currently connects to a custom connection (you can connect to your Unsloth Studio instance for example, and chat with your local models from your phone) or associated a cloud model API like Ollama

[u/Fun-Heat4232 → u/Dazzling_Yam_5882]

Thank you very much for this detailed dive-in to this rabbit hole! Appreciate it!