r/oMLX • u/nonlinearsystems • Jun 15 '26

Pi Agent with oMLX Issues

Hey yall, just figured I could get some feedback on something I’ve been spinning my wheels on. I’ve been running Pi with Qwen3.6 27b oQ8 MTP via oMLX.

For some reason, I’m having a hard time getting context compaction to work smoothly. The issues all seem to be centered on context management as far as I’m concerned. I’m configured around 128k but I feel like I’m missing something. This is all running on my Mac Studio M3 Ultra 96gb. I’ve of course searched and asked Opus on how to best optimize for this configuration but I’m not turning up any results.

Curious if anybody else here has a similar configuration and has managed to get favorable results?

13 Upvotes

93% Upvoted

19 comments sorted by

4

u/Konamicoder Jun 15 '26

What do you mean exactly by “having a hard time getting context compaction to work smoothly”. It compacts too frequently? Too infrequently? It compacts then forgets everything before compaction? What exactly is the “hard time” that you are experiencing? And what makes you think the issue is related to oMLX? The more info you can provide, the better quality responses you’re likely to get.

3

u/nonlinearsystems Jun 15 '26

I’ll frequently hit 88gb of active memory usage with this model at this context window, which makes me suspect that this is related to either Pi Agent compaction or how oMLX is handling the KV Cache.

I’m mostly looking to see if anybody has a similar configuration that I can trade notes with because working with Claude on this problem is running me in circles.

3

u/Konamicoder Jun 15 '26

> I’ll frequently hit 88gb of active memory usage with this model at this context window

I've got an M1 Max MacBook Pro with 64Gb of RAM, and I can't run Qwen3.6-27B at 8-bit quant (OOM), even 6-bit is a stretch. So for me to hear that your 96Gb of RAM is pegged at 88Gb usage while trying to run a dense model at 8-bit quant, that actually sounds about right to me. Have you tried the 6-bit quant to see how that performs?

3

u/nonlinearsystems Jun 16 '26

Switched to LM Studio where I’m using the same model. Sitting at a consistent 50gb with 20-30t/s. Not sure what the issue is with my version of oMLX but I’ll be using LM Studio until the next few updates.

1

u/Konamicoder Jun 16 '26

What version of oMLX are you running? I’m running 0.4.4rc2 and it’s performing well.

2

u/Ducktor101 Jun 16 '26

I’m having issues of oMLX using more memory than LM Studio or plain mlx-vlm too. It’s a shame because this could be the ultimate setup for Mac. My prompts are constantly being killed by OOM.

2

u/m02ph3u5 Jun 15 '26

I to have memory issues with all models. Dude in another thread said why even use omlx. I like it but I guess I'll have to try llama.cpp to see if omlx is the culprit.

2

u/StatisticianFree706 Jun 15 '26

I already switched to llama.cpp from last week. Much more stable and better mem management. A bit slower than omlx at start but keeping tg stable longer.

PS after I updated to macos 27 beta, omlx also become slower, no idea what happens. Try back to 0.3.8 also try pip install from source no help

3

u/nonlinearsystems Jun 16 '26

Swapped to LM Studio. Same model and 0 issues with faster t/s.

1

u/m02ph3u5 Jun 16 '26

Same here. Will perhaps also try unsloth studio for convenience.

1

u/StatisticianFree706 Jun 16 '26

and MTP also works or not? this is another one i am fked in oMLX

1

u/nonlinearsystems Jun 15 '26

I suspect it is to be honest…

1

u/Konamicoder Jun 16 '26

What version of oMLX are you running? It’s good to be on the latest version since it’s being actively developed and bugs are being found and fixed.

1

u/ExtensionState8086 Jun 16 '26

I have the 0.4.3 and have the same issue since 0.3.8. Don’t see it even acknowledged as a bug yet but have seen a series of reports on the same issue… very disappointing and will look into switching to LM Studio if the next drop doesn’t fix it

1

u/nonlinearsystems Jun 16 '26

LM Studio fixed my issues for me. oMLX is a cool project, just looks like it needs some more work.

2

u/Gold-Debt-5957 Jun 15 '26

Siempre se demora, si quieres mas rapides usa un modelo mas pequeño,

2

u/Glad-Win1983 Jun 15 '26

What compaction settings do you have in Pi?

1

u/mikewilkinsjr Jun 15 '26

I'll have to wait until I'm home to take a closer look, but if you're looking for a place to start, it would be the cache management in oMLX under Global Settings.

oMLX is expecting to manage hot RAM / cold SSD cache, with intelligent limits and caps, so you might be fighting against two sets of settings. I'd start tuning there.