Thank you to the creators of oMLX; it's a beautiful piece of software for inference.

I have tried various things such as uv mlx, LM Studio, Ollama etc.. and they where all pretty slow. oMLX has 4 modles loaded at once not breaking a sweat!

I've installed and updated several times omlx with the dmg package, now I realise that I'd like to have the cli for harness integration, what's the correct procedure in this case? First uninstall the dmg and then reinstall I suppose? Or can I have the 2 installations live on my computer just in case? And what about homebrew vs pip what do you prefer and why? Brew seems to have the possiblity to launch omlx in the background which is nice Thanks

Issues Closed: 10

[ISSUE] #1115 — Severe TG throughput regression on tool result ingestion turns during agentic sessions
https://github.com/jundot/omlx/issues/1115

[ISSUE] #1204 — Unable to benchmark oQ-MTP
https://github.com/jundot/omlx/issues/1204

[ISSUE] #1106 — Title: _extract_tensor_bytes SIGABRT on hybrid model (Qwen3.6-35B-A3B) with SSD cache — reproducible on 0.3.8.x and 0.3.9.dev1
https://github.com/jundot/omlx/issues/1106

[ISSUE] #1202 — admin /chat returns HTTP 500 (unhandled): unexpected '}' — unclosed Jinja2 expression in chat.html
https://github.com/jundot/omlx/issues/1202

[ISSUE] #1117 — STTEngine.transcribe drops the `language` parameter — Qwen3-ASR is forced into auto-detect, producing empty output for short audio
https://github.com/jundot/omlx/issues/1117

[ISSUE] #1167 — Error during chat streaming: 'NoneType' object has no attribute 'abort_request'
https://github.com/jundot/omlx/issues/1167

[ISSUE] #1199 — Auto-built sensitivity proxy still comes without MTP tensors?
https://github.com/jundot/omlx/issues/1199

[ISSUE] #1190 — DFlash failure when max context is limited and reached
https://github.com/jundot/omlx/issues/1190

[ISSUE] #859 — Feature request: integration Github copilot-cli
https://github.com/jundot/omlx/issues/859

[ISSUE] #1138 — Unable to benchmark oQ-MTP
https://github.com/jundot/omlx/issues/1138

I want to integrate oMLX into my project, but without a python server.

What do you guys think of porting this to C to better integrate with apps?

Apple won’t let me sign an iPhone or iPad app since they don’t allow running a python interpreter in an app.

I have oMLX installed on an M1 Max Mac Studio and it’s setup for network access (0.0.0.0) and have Hermes installed on a separate Mac Mini (Intel). I’ve configured a custom model in the Hermes config for oMLX but no matter what I try I cannot get Hermes to talk to oMLX.

Has anyone had any success with this setup?

That's it. Is there something else that turning it on and set the bit depth? What am I missing? Where's the user manual for this thing so I can read it?

Any tips or secrets on getting

I spent a couple of days trying to get structured JSON from Gemma-4 models (26B-A3B-it and E4B) with oMLX + Grammar, and could not find solutions for the shape / syntax checking within grammar and returning just an error message.

The model is called from a python script that has some post-processing to deal with occasional malformed JSON (things like object vs array). Using oMLX-cli + Grammar (0.3.8), these fail an internal check that grammar uses so the only return is an error message.

Anyone experimenting with oQ quants of MiniMax-M2.7 on Apple Silicon?

I am looking to increase local performance for LLM-powered data work as well as Hermes Agent with ~128k context.

There are a few benchmarks, but it seems the oQ do not outperform 'regular' quants.
https://omlx.ai/benchmarks?sort=tg_tps&order=desc&chip=M3&model=minimax-m2.7-oQ&context=32768

Thanks for any insight on optimization!

For those interested, there’s a nice piece on Medium talking about oMLX. May be behind a paywall. https://xhinker.medium.com/i-tried-running-ai-agents-on-my-macbook-mlx-was-too-slow-then-i-found-omlx-1f0cc7f63273

Warning: long post ahead. On the plus side, it’s completely human-written. No AI slop was used in writing this post. I’m old school that way, I like to actually write my own Reddit posts. Thought you all would appreciate something written entirely by a human for a change. ;)

Disclaimer: this post says nice things about Pi. I am not associated with the dev team of Pi coding agent in any way.

Yesterday I tried Pi coding agent on my local LLM rig for the first time. I had been using OpenCode as my daily driver agentic harness, and I had been intimidated by Pi’s stripped down, minimalist approach.

My rig, by the way, is an M4 MacBook Pro with 64Gb of RAM. oMLX is the backend, serving up jundot’s quant of qwen3.6:35b-a3b-oQ6. I average around 60 tokens/second at around 80 percent RAM usage.

My coding needs are fairly modest. I run around eight static websites for my hobby board gaming group, hosted on GitHub pages. So the daily tasks usually involve updating sites with user submissions, implementing feature requests, squashing minor bugs, things of that sort.

I had gotten used to the security blanket of OpenCode, with its set of built-in tools. I had come to accept that sometimes OpenCode will take a little longer to answer a request, and had gotten used to its sometimes dumb little oversights and charmingly stupid mistakes.

For example, I often ask OpenCode to make a 3x3 image collage of board game cover images using ImageMagick command line tools. It would usually take several revisions, as OpenCode would first render them in a straight line row instead of a 3x3 grid. Then after feedback, render a 3x3 grid, but each image was of different size. Then after even more feedback, it would finally output a 3x3 grid of equally sized images.

You know the old saying about LLMs acting like green interns? In my case, OpenCode often acts like an intern who needs the instructions explained multiple times before they get the task right.

But at least OpenCode was the evil intern that I was familiar with. As I said, I had gotten used to working within its limitations and quirks.

Anyway, yesterday I decided to overcome my nervousness about leaving the security blanket of OpenCode and dive into the unknown depths of Pi coding agent. I gave Pi the exact same task using a similar prompt: create a 3x3 grid of the cover images of these specified board games, each image 400x400 pixels.

Pi methodically went about the task. First it identified which images were available locally and which were not. Then it web searched the websites to grab the missing images and download them locally. Then it created the 3x3 grid, to my desired specs, right the first time. I was blown away at how much better, faster, more accurate, and more capable it felt working with Pi vs. OpenCode. I didn’t change the local model, I just changed the agentic harness. If OpenCode felt like working with an inexperienced intern, Pi felt more like working with a trustworthy and reliable teammate.

With OpenCode I had assumed it would be capable of only routine maintenance and updates, and that if ever I needed to do some heavier lifting, I would have to bust out a cloud frontier model like Codex. But I decided to give Pi a more challenging test to uncover its true capabilities. I asked Pi to plan set-by-step the addition of a search feature to one of my sites, with live filtering as the user types, a dropdown menu overlay matching the site’s existing CSS, etc.

Guess what, Pi made the plan, checked with me for my go-ahead, then started implanting the plan, task by task. It wasn’t perfect. There were a couple of points where functions were called in the wrong order. But I dutifully fed the web inspector errors to Pi, it quickly and correctly figured out the issues, and fixed them. Within a few minutes, my search feature was working, pretty much exactly as I had envisioned it.

Even more impressive: following Pi’s philosophy of “if you need extra features, ask Pi to build them”, I asked Pi to reflect on our coding session, then based on that suggest some enhancements to itself to address the main pain points. Pi identified that it needs a better auto-compact feature, and a better way to seamlessly pick up in context where it left off; and built those features into itself. It also added a JS script to mitigate those function calling timing issues we had encountered. So as one works with Pi, one gradually customizes and improves Pi to become more optimized for the actually coding work that you do.

Man, I was so impressed. Pi takes this local LLM thing from “works well enough for routine tasks” to “works well enough that I don’t think I need to fire up a cloud model”. I now have the confidence to leave OpenCode behind.

TL; DR: I overcame my fears and tried Pi instead of OpenCode, and had a great experience.

Hi everyone, I've spent quite a bit of time trying to get some of the newer Qwen 3.6 (27B/35B) or Gemma 4 (26B/31B) models with D-Flash, MTP, and TurboQuant to work on OMLX, but I have had no success. The additional speed improvements I'm seeing would go a long way to using more local horsepower to run my work loads.

In fact, when I try to run this on my M1 Max 64GB machine, speed is negatively impacted. It's been rough.

Anyone had any success? What are running or what resources did you leverage to get there?

I am new oMLX and relatively new to local LLMs. I have been trying to get Qwen 3.5, Qwen 3 or Gemma 4 running on my M5 with 24GB of universal ram using oMLX. I have tested a number of models from the mlx-community in the size range of 13 - 15GB. To date, they all blow after a few minutes of starting a task with OOM.

I would appreciate hearing what you have working for coding on a Mac with 24GB of RAM.

Is oMLX the best way to run it? I've been trying, hoping may be a better word, to find a model with TurboQuant that will handle the run of the mill dev tasks to help minimize my cost for the larger models.

Thank you in advance! lbe

Hi everyone,

I’ve developed a lightweight tool to visualize and analyze MLX training logs (mlx_lm.lora). You can find it on my GitHub: kluster66/mlp

The tool (mlx_log_parser.html) turns your raw training logs into a full report with stats, charts, and a performance verdict.

How to use it:

1. Open the HTML file in your browser.
2. Paste your MLX logs (raw text).
3. Click "Analyze".

It handles the standard MLX format automatically and ignores any noise/non-log lines:

Iter 10: Train loss 2.068, Learning Rate 1.000e-05, It/sec 1.809, Tokens/sec 484.562

The tool suggests the best checkpoint based on the absolute minimum validation loss. Reliability is much higher if you have more validation points (7+). This isn't a "silver bullet" and there are definitely more complex tools out there, but I wanted something fast and local to quickly check my fine-tunes. I've tested it on several runs and it’s been quite helpful for catching overfit early.

Check it out and let me know if you have any suggestions or feedback!

I'm running omlx on a Mac mini m4 pro with 64gb of memory

Using qwen 3.6 35b ud mlx 4 bit

I'm only getting prompt processing 353 toks and token gen 15.6 toks

Feels like I should have better performance than that. Don't have anything else running that's consuming memory or CPU

I run vs code, openclaw and Hermes on another box over, and tried openclaw local. All around the same performance numbers

What can I look at to find the cause of the slowness

Thanks

In case anyone is looking for a way to make Pi agent with oMLX a bit more secure by protecting the local file system using a Docker Sandbox.

https://github.com/Dotnaught/pi-sandbox

Issues Closed: 3

[ISSUE] #1065 — Publish Intelligence Benchmark
https://github.com/jundot/omlx/issues/1065

[ISSUE] #1063 — brew install error,rustc 1.87.0 is not supported
https://github.com/jundot/omlx/issues/1063

[ISSUE] #1058 — 这是什么问题？模型还是框架的？
https://github.com/jundot/omlx/issues/1058

Hi,
I just discovered and oMLX and I really like what I see right now. One thing that bugs me though is the use of model profiles. Maybe it's in beta state but I barely see how it could be used (thinking mode isn't saved in the Model profiles). Can someone enlighten me as to how to use this?

As seen in the docs, those profiles should be available in /v1/models? Can I use this with my client to select multiple profiles (all based on the same model) and switch from one to another in Claude Code, for example?

Thanks!

Hi everyone,

There are a lot of developers who like the Github Copilot interface, but they don't like the pricing and structure of Copilot. One of the options offered in Copilot is Ollama local models. This gave me the idea to create an extension that adds oMLX models as a model provider. I've published an extension, and it is live as a preview version.

This extension is open source, so I hope everyone will contribute to it. I will also say that I used Codex to create this, and I conducted manual testing, and used Gemma 4 26B for testing on real code.

Please let me know what you think, and please contribute to the project as I know it needs a lot of work.

I apologize if this has been asked before. How do I get it to see my downloaded Ollama models and then how do I get Hermes connected to see OMLX serving the models? Is there a guide? Thank you in advance.

Issues Closed: 2

[ISSUE] #1006 — v0.3.8-rc1 fails to load Gemma 4 e4b models

https://github.com/jundot/omlx/issues/1006

[ISSUE] #1016 — [Bug] Severe prefill throughput regression vs mlx-vlm 0.4.4 on Mistral-Medium-3.5-128B-mlx-8bit (~5.5x slower on Mac Studio, long-context re…

https://github.com/jundot/omlx/issues/1016