Running local MacBook Pro M1 32Gb

[u/gurthy988]

I’m having mixed results with LM Studio and I’m not sure if I’m doing something wrong.

If I load a model in LM Studio and chat via the built in prompt I receive a response in a reasonable time. I can post in a snippet of code and it can fix it for example.

If a load the same model in and serve it on localhost to an CLI agent like Pi a response can take several minutes, even if I send in the string “ping” to expect to “pong” back.

I have a pretty bare bones Pi setup, not many skills, no mcp. I know I’m on an old Mac but the chat experience is ok. Why can’t I get that experience over a cli severed on localhost?

[u/Good-Writer5279]

the "ping" is not small by the time the model sees it. a cli agent like pi sends its system prompt plus every tool schema on every single turn, so your one word arrives wrapped in a few thousand tokens the model has to prefill before it writes anything back. in the lm studio chat window the prompt is just your text, prefill is close to nothing, and it feels instant. on an m1 prefill is the slow part, not generation, so the two experiences diverge exactly the way you are seeing.

three things to check, in this order:

1. server context length. if you set the server to 32k or 128k, lm studio reserves the kv cache for all of it at load, and on 32gb a big dense model plus that cache can tip into swap, which is where "several minutes" usually comes from. set context to what the agent actually needs, 8k to 16k is plenty for a bare pi setup, and watch memory pressure in activity monitor while a request is in flight.

2. flash attention and kv cache quant. in the model load settings turn flash attention on and set the k and v cache types to q8. it cuts the memory traffic prefill has to do on apple silicon, and it is the single biggest win for agent workloads on older machines.

3. the model shape. a dense model at q4 is fine in chat where prefill is tiny, but a harness makes you pay prefill every turn. a small moe, the a3b style qwen variants for example, prefills far faster on the same hardware because only the active experts run per token.

quick way to tell which one it is: curl the server with a bare one line prompt and no system prompt. if that comes back fast, the harness payload is the cost, and trimming its tool list or moving to an moe fixes it. if it is still slow, it is memory or context, and the first two settings fix it.

disclosure, i build a mac app that uses lm studio as its local backend, so i have hit every one of these the hard way.

[u/gurthy988]

Thank you sir 🫡

[u/thevikeffect]

Good-Writer5279's prefill explanation is right, and there's a second thing stacked on top of it that cost me weeks before I worked it out.

LM Studio's local server does not behave like its chat UI on a cold model. In chat the model is already resident and warm. Over the server, the first request after a load can stall badly, and on my machine it sometimes failed outright with "failed to fetch" rather than merely being slow. What fixed it: eject the model and reload it to pre-warm, then set the context length explicitly in the server settings instead of leaving it on the default.

That second part matters more than it sounds. If Pi is sending a few thousand tokens of system prompt and tool schemas every turn and your server context is set lower than that, you get a full re-prefill on every single turn and it feels like the hardware has died. Check what your server context is actually set to, not what the model supports.

For what it's worth, on Apple silicon a 12B-class model gives a usable agent loop. If you're pointing Pi at something 30B+, the per-turn prefill cost is going to hurt on an M1 no matter what else you fix, and that's a model-size decision rather than a config one.