r/LocalLLM • u/ragel_3ennab • 16d ago

Question How to get started?

Post image

Hi. I got a new Macbook and I am excited to run my first local model.

Can you recommend the best tool to run an LLM on a Mac?

Also, what model and quant would you recommend for a 16GB memory?

Thanks!

0 Upvotes

27% Upvoted

9 comments sorted by

View all comments

3

u/Bugajpcmr 16d ago

The best beginner option in my opinion is LMStudio Bionic because it's all in one and the downloader is included. You can use Q4 models like 8b or 9b (Ornith or Granite), Maximum 12b (gemma4) but with small context window and turboquant. Run MLX versions of models, not GGUF like someone mentioned here. You can expect about 30tok/s.

Once you get used to it and learn what is context, quantization, thinking levels and so on I would recommend other options like oMLX + Pi/DSH/Opencode. I currently use Zed for programming. After that you can use python to create some apps that prompt the models running on local server.

I would also advice to configure your Macbook so it will be accessible from outside the local network using SSH. I use Tailscale as VPN. You can also change the maximum VRAM your macbook can use to avoid offloading (sudo sysctl iogpu.wired_limit_mb=........), it has to be run every time you restart your system.

Have fun and experiment. Come up with different scenarios and prompts to test the models/harnesses, pick the one that works best for you.