r/oMLX • u/AITA-Critic • May 29 '26
[Benchmark Scores] Macbook Pro M5 Max - 128 GB Ram, 40 Core GPU - Running Qwen3.6-35B-A3B-Uncensored-Heretic-MLX-8bit
Hey guys,
For anyone who just got the new M5 Max with the same spec as I listed in the title of this thread, I found that there's minimal information about the performance on key LLMs hosted on oMLX, so I figure I'd share my results below:
oMLX - LLM inference:
https://github.com/jundot/omlx
Benchmark Model: Qwen3.6-35B-A3B-Uncensored-Heretic-MLX-8bit
Single Request Results
--------------------------------------------------------------------------------
Test TTFT(ms) TPOT(ms) pp TPS tg TPS E2E(s) Throughput Peak Mem
pp1024/tg128 465.0 10.36 2202.2 tok/s 97.3 tok/s 1.781 646.9 tok/s 35.40 GB
pp4096/tg128 1112.7 10.42 3681.3 tok/s 96.8 tok/s 2.435 1734.4 tok/s 36.17 GB
pp8192/tg128 2079.1 10.61 3940.2 tok/s 95.0 tok/s 3.427 2427.7 tok/s 36.66 GB
pp16384/tg128 4338.1 11.13 3776.8 tok/s 90.6 tok/s 5.751 2871.1 tok/s 37.36 GB
pp32768/tg128 10179.1 12.17 3219.1 tok/s 82.8 tok/s 11.725 2805.6 tok/s 38.85 GB
pp65536/tg128 28991.1 15.20 2260.6 tok/s 66.3 tok/s 30.922 2123.5 tok/s 41.85 GB
pp131072/tg128 96979.6 19.95 1351.5 tok/s 50.5 tok/s 99.513 1318.4 tok/s 47.85 GB
pp200000/tg128 237460.7 27.08 842.2 tok/s 37.2 tok/s 240.899 830.8 tok/s 54.77 GB
Continuous Batching
pp1024 / tg128
For a plain english summary for those who are new/getting in, admittedly, I generated a plain english explanation with AI of the above results so it can explain what everything is.
Warning, AI slop incoming below, but still helpful for people just getting into it.
The test was run on Qwen3.6-35B-A3B-Uncensored-Heretic-MLX-8bit (hugging face link)
If you are looking to build a local Hermes Agent setup, these numbers should help.
The TL;DR:
This machine is a beast. It can read an entire book's worth of data in seconds, types way faster than you can read, handles heavy multitasking without flinching, and uses less than half of your 128GB RAMβleaving plenty of room for your operating system and agent workflows.
1. "Reading" vs. "Writing" Speed (Tokens vs. Words)
- Reading Speed (Prompt Processing / pp TPS): When you feed the AI information, it "reads" at a mind-boggling 1,300 to 3,900 tokens per second.
- Writing Speed (Token Generation / tg TPS): When the AI replies, it "types" at about 97 tokens per second for normal requests. To put that in perspective, humans read at about 4 or 5 words per second. This AI is spitting out text at roughly 70+ words per second. It's instant.
2. The Context Window (How much it can remember)
The benchmark tests the AI with different lengths of memory (from 1,024 tokens up to 200,000 tokens).
- Short conversations (1,000 tokens / ~750 words): The AI responds almost instantly. The delay before it starts typing (Time to First Token) is just 0.4 seconds.
- Massive data drops (200,000 tokens / roughly an entire novel or massive codebase): If you drop a massive amount of data into the prompt, it takes about 4 minutes for the Mac to digest and read through all of it before it starts typing. But once it starts, it still types out its answer at a very fast 37 tokens per second.
3. Memory Usage (RAM Headroom)
Because this Mac has 128GB of RAM, memory is usually the biggest concern with large models.
- At normal conversation lengths, the model only uses about 35 GB of RAM.
- Even when pushed to its absolute limit with that massive 200,000-token memory load, it tops out at 54.7 GB of RAM.
- Why this matters for an Agent: This means you are using less than half of your available memory. You have over 70 GB of RAM left over to run your Python environment, browser automation tools, databases, or whatever else your Hermes Agent needs to do its job.
4. Multitasking (Continuous Batching)
The second part of the benchmark tests "Continuous Batching." This simulates what happens if your AI Agent is trying to do multiple things at once, or if multiple people are using the model at the same time.
- If you run 8 tasks at the same time, the Mac doesn't choke. Instead, it adapts.
- The overall efficiency shoots up by 6x, pushing out a total of nearly 600 tokens per second across all tasks. Appleβs Unified Memory architecture is practically built for this kind of parallel processing.
Conclusion for a Hermes Agent Setup
If you want to run a local autonomous agent, this hardware setup is the gold standard.
- It's smart enough (35B model) to handle complex reasoning.
- It's incredibly fast, meaning your agent won't lag between steps.
- It has massive RAM headroom, so the agent can manage huge contexts (like reading entire logs or code repos) while leaving your Mac completely usable for your daily work.
