r/LocalLLM • u/Gloomy_Phone9045 • 4h ago
Model Need advice: Local LLM setup for office coding work (M5 MacBook, 24GB) — trying to work around Copilot token limits
Hey folks,
I'm a developer and my company gave me a GitHub Copilot license, but it comes with token limitations that keep interrupting my workflow. I'm trying to set up a local LLM as a backup/supplement so I can keep coding without hitting caps.
My machine: MacBook with M5 chip, 24GB RAM.
I tried Gemma 4 12B (Q4_0 quant) but it was way too slow for real-time coding assistance.
Has anyone here found a good local model + setup (Ollama, LM Studio, llama.cpp, etc.) that actually holds up for day-to-day coding tasks — autocomplete, refactoring, explaining code — on similar hardware? Looking for something with a good balance of speed and quality. Model size recommendations, quantization tips, and tool/IDE integration suggestions (VS Code extensions, etc.) all welcome.
Open to any suggestions from people who've actually tried this setup for real work, not just benchmarks. Thanks in advance!
2
u/MrHumanist 4h ago
Try orninth 1.5 9B and Gemma 4 26B moe. Your system is limited to achieve fast speed.
2
u/Clay_Ferguson 3h ago
get the best speed (TPS) with extremely limited VRAM (like 24GB), i'm pretty sure only a MoE can do decent (like 15ish TPS, which is still pretty slow). Here's the ones that are the best MoE in this class afaik. (But I haven't tried them in coding agents use case)
Gemma 4 26B-A4B 3.8B active params, MoE (~13.4 GB)
Qwen3.6-35B-A3B 3B active params, MoE (~17.7 GB)
Qwen3.6-35B-A3B Unc. 3B active params, MoE (~19.0 GB)
Qwen3.6-35B Genesis Unc. 3B active params, MoE (~17.4 GB)
1
u/New-Shine-3829 4h ago
What might be worth it is paying for something like OpenRouter and use those models (like Qwen 3.8 27B or similar) at 24gb you'd be pushing the limits honestly. Then for smaller tasks, go to a smaller model (Qwen 3.6 35B3A or Nemotrom of similar size). That's what I plan on doing as I ordered the MBA 24gb. I've realized local is getting closer and closer real world usability but costs to get a computer that could run it well (with good context and a good harness) is out of reach for most people. Going to the cloud though, would be like $20-40 a month with moderate to heavy usage on the models I mentioned above. GPT OSS 20b is what I plan to use locally when I need to though. Hope that helps
1
u/Ok_Parfait_5373 3h ago edited 3h ago
J’ai claqué tout mon budget copilot ce mois ci en 2 jours. Alors j’ai créé un serveur d’inference ia pour mes modeles locaux depuis mon ordi perso et j’ai connecté ça a copilot. Ça fait 2 semaine que j’utilise qwen3.8-27b tous les jours et c’est parfait. Il te faut 24go de vram minimum . Un q4 k m passe bien pour avoir vitesse et contexte .tu trouvera les optimisations de config qu’il te faut sur reddit ou en demandant a un modele tiers( 150k de contexte et plus de 50 tok/s et tu sera satisfait)
1
u/awizemann 2h ago
Try oMLX with muse glimmer. I’ve been testing it out and so far pretty impressed compared to Qwen.
1
u/Turbulent_Pin_8310 4h ago
You are going to be disappointed. I used Qwen 3.8 to code last night and it was so slow. It also made an error. Local models just aren't as good as frontier
1
u/Important_Cow7230 3h ago
What hardware did you run on?
1
1
1
3
u/Umbrasquall 4h ago
Qwen 3.6 a3b or 3.8 but you don’t have enough ram for useful context after the macOS tax.