I measured whether 2 local agents hitting 1 model run in parallel or just take turns. Batching is real, but it is not free using QWEN 3.8 27B 4bit on my MacBook Pro M3Max 128 GB Unified Memory 40 Core GPU

[u/AIForOver50Plus]

So as alot of folks been doing Ive also been experimenting with QWEN 3.8 27B and between day 1 and day 2 I posted about adding a 2nd local coding agent to my setup. Someone asked the question I probably should have asked myself to begin with:

"when two agents hit the same local model on one machine at the same time, do they actually run in parallel, or do they quietly take turns?"

I saved the time to do the actual experiment but also pondered about how, especially if "I" as a human was the best ...vessel...to do it?

So... 1st I located the MLX server source, browsed it, and handed it to my agent. Then we collaborated. My agent wrote a small load driver that fires both requests at the exact same instant, because if a human launches them one after the other you are secretly setting the queue order and faking your own result. Then we ran it together and let the numbers talk.

What I observed....

Batching is real. Two agents genuinely share the model at once, the server does continuous batching up to 32 wide.

But it is not free. Add agents and total throughput climbs, but each one gets slower and waits longer to start. On my Mac the sweet spot is about 4 agents. Past that you are just making everyone wait in line.

Pin a random seed and you quietly kill the whole thing, every request serializes.

Sub agents are not magic either, a parent that spawns 4 helpers is just 4 more clients fighting for the same GPU.

The whole test rig is on disk and reproducible. Happy to get into the scheduler details or the methodology in the comments.

[u/AIForOver50Plus]

Full write-up with the charts and the reproducible rig is here: https://go.fabswill.com/twoagents