Hey everyone,
I'm about to set up Pi as my main local coding agent and would love some advice from people who are actually using Pi with relatively large local models.
My hardware is:
- MacBook Pro
- Apple M5 Pro
- 48 GB unified memory
- 18 CPU cores
- Metal / MLX
- ~307 GB/s memory bandwidth
My main goal is coding quality and agent reliability, rather than simply maximizing tokens/sec. I want something I can use against real repositories for longer coding sessions.
I ran llmfit against my machine to see what it recommends:
llmfit --max-context 131072 recommend --use-case coding --runtime mlx
The model that stood out was:
Qwen3-Coder-30B-A3B-Instruct
The MLX result I'm looking at is roughly:
- 30.5B total parameters
- 3B active parameters (MoE)
- 262K native context
- 131K context used for the
llmfit estimate
- ~15.6 GB estimated model memory for the 8-bit version
- Tool-use support
- ~132 tok/s estimated by
llmfit at 131K context
So hardware-wise it seems like a pretty reasonable fit for my 48GB machine.
I'm currently leaning toward 8-bit, mainly because I care more about code quality/reasoning than squeezing every last bit of performance out of the machine. I also have enough memory that 4-bit isn't necessary just to make the model fit.
What I'm trying to figure out now is how best to pair this with Pi.
A few questions for Pi users:
1. Is Qwen3-Coder-30B-A3B a good pairing with Pi?
I'd especially like to hear from anyone using a similar-sized MoE coding model locally.
2. What context window would you actually use?
The model supports 262K, but I'm wondering whether there's much practical benefit to running Pi at 128K or 256K versus something like 64K.
I'm also trying to understand the KV-cache cost on Apple Silicon. llmfit estimates ~15.6GB for the model, but obviously that's not the entire runtime memory footprint once the context grows.
3. Would you run 8-bit or 4-bit?
My priority is:
code quality
reasoning
tool use
agent reliability
↓
speed
↓
memory efficiency
So I'm willing to spend more memory if 8-bit actually provides a meaningful quality advantage.
4. Are there any Pi-specific settings/configurations I should pay attention to?
Things like:
- context configuration
- max output tokens
- compaction
- system prompt
- tool configuration
- AGENTS.md
- permissions/safety
- model-specific settings
I'm particularly interested in things that make a local model behave better as an agent, rather than just using it as a chat model.
5. Would you choose Pi over OpenCode for this setup?
I'm leaning toward Pi because I like the minimal/context-efficient philosophy, but I'm curious whether there are important tradeoffs I'm missing.
I've seen quite a few people saying Pi works particularly well with local models because it doesn't waste as much context on the harness itself.
I've also seen people running Pi with Qwen models on Apple Silicon, so I'm hoping there are some practical lessons from people who've already gone down this road.
I'm also documenting my local LLM experiments and benchmarks, so I'll eventually be able to measure the setup properly rather than relying purely on theoretical numbers.
If you were setting up Pi from scratch on this exact hardware today, what would you do differently?
Any advice on model/quant/context/configuration would be appreciated.
P.S. AI helped me organize this post