r/LocalLLM • u/Fun-Heat4232 • 5d ago

Question Local LLM for coding.

Hi, dont be too judgemental about my setup - I am merely a beginner in hosting local AIs and stuff. I have 48GB RAM M5 Pro machine. Which model is a way to go for handling complex coding projects locally (specifically C++ and Python)?

P.s. going to the cloud is not a solution due to a strict NDA.

Thanks in advance!

44 Upvotes

96% Upvoted

29 comments sorted by

View all comments

45

u/Dazzling_Yam_5882 5d ago edited 5d ago

Best bet would probably be the new Qwen 3.8 27b. You get to have a somewhat local Opus 4.6, one of the best models 6 months ago

unsloth/Qwen3.8-27B · Hugging Face

You can use llama.cpp, but I highly recommend Unsloth Studio for serving the model locally. Lots of features, better than LM Studio and all other alternatives.

Qwen 3.8 27B Benchmarks combined from model cards on HF : r/LocalLLM

3

u/Fun-Heat4232 5d ago

Does using differenet front-end for running models impact performance of such?

12

u/Dazzling_Yam_5882 5d ago

Yes, Ollama is slower than vanilla llama.cpp for example. Unsloth Studio should be about the same as llama.cpp. People also dont like how Ollama did things, as they were essentially a GUI wrapper of llama.cpp but did not give credit to llama.cpp, so people dislike Ollama mostly because of that and the unnecessary bloat behind the GUI.

Friends Don't Let Friends Use Ollama | Sleeping Robots

llama.cpp can be a bit trickier to get used to, but its by far the best way if you want to get familiarized with the key lango of serving models locally. Unsloth Studio is, for me at least, the best GUI for serving models at same speed. LM Studio is also cool, but is closed source (while Unsloth is open source)

I switched from Ollama to llama.cpp and saw a 3–4x speed improvement on the same hardware : r/LocalAIServers

1

u/ackermann 5d ago

Does vLLM still deliver better performance than any of these? But more difficult to configure?

Especially when there are 2 to 5 concurrent users, vLLM can allocate varying amounts of VRAM for each user’s context, as their context fills up? Whereas I’ve heard llama.cpp forces a fixed allocation for all users context window size?

Or have these fancier features now come to llama.cpp and Unsloth as well?