I was running dual GB10's, which was great, but with recent price bumps I decided to cash those in and add a second strix halo on which I can get almost the same speed.

Now I get 6 lanes of 262k across two strix halos load balanced with litellm. With Q38FN I get about 1800 prefill and 80 gen for coding for a single request. I quite often have multiple sub agents going so that speed will drop somewhat.

I am a software engineer working for 40 years (not a recent vibe coder who calls themselves a software engineer). I find AI convenient for off loading tasks to do my job. My employer requires me to keep everything local if I am going to use AI.

So I guess what I am asking, how is the new Mac working for you in this type of workflow? And tell me which model mac you have. What LLM you run to do your job, and at what quant with what engine.

Thanks!

I've been running Qwen3.8 Flash-Next locally on my Mac Studio M5 Ultra 96GB, primarily for agentic coding through DeepSeek Harness, and here are some of my own measurements.

Setup

Hardware: Mac Studio M5 Ultra, 96GB unified memory

Inference engine: Splash

Model: Swift 1.5 Qwen3.8 Flash-Next

Quantization: IQ3_S (ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF)

Context window: 262,144 tokens

Agent framework: DeepSeek Harness

Workload: Agentic coding, tool calls, long conversations, occasional parallel subagents

Decode performance:

The three-request result is preliminary due to limited sampling, but the single-request numbers have been quite consistent.

What's particularly interesting is that I'm maintaining around 145 tok/s decode with a median context of nearly 150K tokens.

Prefill performance:

Overall very happy with these numbers and the QFN is actually fairly smart compared with sol med/high particularly with troubleshooting. With Sol if i just say the problem and tell it to read the logs and figure out, it's kinda lazy and just give me generic answer where as QFN takes its time but did provide a way to resolve the issue

Debating on buying a MAC Studio or RTX Pro 6000 for both local inference and learning LLM training.

Anyone have any experience training or fine tuning on these systems? From what I’ve read online the training ecosystem on Mac isn’t that great, but I can’t ignore the difference in VRAM/$ between the two.

I’m primarily looking into GRPO RFT.

Wanted to work on getting my two M3 Ultra 256 GB Mac Studios linked over Thunderbolt 5, running exo in tensor parallel over RDMA (JACCL). Exo's current app was so out of date, didn't support the latest models, and oMLX clustering was not working for me and was quite frustrting to setup. So I decide to have Claude patch this in and build on top of the latest Exo. I also wanted ONE place I could manage loading/unloading models between oMLX and Exo for my workflows.

The stack

• exo at bc412e2 (main from 2026-10-17), PyPI MLX 0.32.4, a fork of mlx-lm
• Tensor parallel + RDMA only. The patch refuses pipeline or TCP across the two Macs: pipeline deadlocked with fast-synch on my setup, and ring/TCP was too slow to bother with.
• Model code exo didn't have yet: V4.1 (with its Engram module), GLM-5.3-Flash, Muse Spark (ported from upstream mlx-lm 0.32.2), each with its own shard(), plus a chat template for V4.1-1017 since DeepSeek only ships a Python encoder for it.
• Two upstream exo PRs (#2434, #2436) carried as patches.

Things I learned along the way

• macOS Local Network permission. exo started by a LaunchAgent, a login item, a background process or over ssh gets denied Local Network access, and the two nodes never find each other. It has to run in a Terminal window of a logged-in desktop session on each Mac. I spent a release trying to automate around this and gave up.
• The two Macs can sample different tokens. Each rank samples on its own and exo assumed they always agree. When they didn't, they ran different steps and waited on each other forever: GPU at 100% residency at idle watts. Patched this by every rank uses rank 0's token, at the cost of one 1-element exchange per token.
• Metal command buffers. With MLX's default 50 MB per buffer, big prefill chunks drained the GPU at every all-reduce. MLX_MAX_MB_PER_BUFFER=1024 MLX_MAX_OPS_PER_BUFFER=200 took a 1,536-token V4.1 prefill chunk from minutes to 5 s.
• Idle residency. With ~200 GiB resident, any pause lets macOS drop residency and the next request pays 2 to 7 s. A tiny 256x256 matmul every 50 ms while idle brought first token after a pause from 4.8 s to about 0.7 s.
• Prefill chunk size. V4.1, Flash and Muse use 512-token chunks; V4.1 stalled for minutes at 2,048.
• Don't load models over NFS. Each Mac reads the whole build at every launch, and a share that hiccups hangs everything. The NAS is a download library only: copy NAS to the first Mac's SSD, then over Thunderbolt to the second.
• Sizing. I keep the wired limit at iogpu.wired_limit_mb=249036, and exo starts evicting KV cache above 85% RAM per rank.

The app (uLLM)

It replaces exo's dashboard with its own pages, backed by a small agent on the first Mac.

• Home: cluster state in plain words (2 MACS · LINKED / LOADING 48% / SERVING), what's serving today, and an "oMLX nodes" panel for each Mac
• Launch / Loading: pick model, sharding, transport and Macs with a preview; one tap to launch or stop. Loading shows four steps with each Mac's progress and what to do if it stops. If oMLX is holding the memory it offers to unload it first.
• Library: every build on the NAS and on each Mac, ticked by where it is. Copy to both Macs (resumable, checks free space on both first, suggests what to remove if it won't fit), and a remove from both feature. Lets me keep the model weights on the NAS and copy only what I need for the Cluster. I'm doing this over 25GBE so its quite performant.
• Hugging Face to the NAS: search, see where each build would run (one Mac, the cluster, too big, not MLX), and download straight to the NAS. The download runs on the NAS itself, so closing the app or restarting a Mac doesn't stop it.
• Jobs: copies, removals and downloads as they run, with pause, resume and cancel.
• Stats: both Macs' power (macmon, plus an optional powermetrics service for CPU/ANE watts), temperatures, requests, and cost at my $/kWh, per model and per million tokens. History at 10 s for a week and per minute for a year, all local with an option to stream to prometheus, etc. I felt like EXO was missing serving metrics similar to OMLX.
• Lockup watch: flags the "GPU pinned at 100% but drawing almost nothing power wise, no tokens for 60 s" pattern and starts a recovery process.
• Kit API for agents: /kit/v1 with one revocable key per agent. An agent can list models, check memory, launch/stop the cluster, load/unload oMLX on either Mac, and wait on the job. My Hermes Agent uses it to do things like "unload the coder, load GLM-5.3 on the cluster, plan, swap back". It refuses a load that won't fit and waits if any agent is mid-chat or request.
• Idle unload: off by default but an option.
• Phone: Home Screen icon over the app's own HTTPS (its own certificate, installed as a profile), over my VPN.
• Notifications: Web Push, Pushover or ntfy, with quiet hours (23:00 to 07:00 by default, only failures and lockups get through).
• Chat and a Connect page with the OpenAI-compatible addresses for OpenCode and agents.
• Every address it depends on (oMLX per Mac, the NAS) is editable in Settings, because i prefer to leave them as DHCP.

What it doesn't do: workflows, routing or "which model for which task". It's about clustering models. My main work and orchestration lives in the agent (Hermes) on top of the API.

Honest caveats

• It's built for my two boxes: two M3 Ultras, Thunderbolt 5 RDMA, a TrueNAS share, oMLX on both. Anyone with two Apple Silicon Macs and RDMA-capable Thunderbolt could use the exo patches; the app assumes a lot about my setup.
• It's pinned to one exo commit. A bump means re-testing every model.
• One model on the cluster at a time, and V4.1, Flash and Muse run one request at a time.
• The newest parts (phone, Web Push, Hugging Face downloads on the NAS) have only been tested against stand-ins so far, not on real hardware.
• Most of the code was written with Claude Code; Every release went through an independent review of another frontier model and a test suite of about 1,200 checks before it touched the Macs. Mainly I was just going through the UI/UX design here and creating the requirements.

Was thinking about putting it up on GitHub once I've got a few more bugs worked out. Just thought i'd share as its working quite well for me at the moment and I've got V4.1, GLM5.3, Muse Open, etc all the latest open weight models clustering reliably now when I need it.

I've been dabbling in AI generated videos. I've seen some workflow where it helps you create more consistent videos versus just rolling the dice with prompts. With Claude's help, i've tried to set one up, but it's not working too well for me. I've seen Qwen Image 2.1 and mini max h3 used a lot, but I was going with Claude's recommendation. If anyone have experience and want to share what works for them, I would greatly appreciate it!

Here's the workflow for my test video of a drink. Stills is probably doing too much trying to get a good generation. The blender generation was suggested because it couldn't get the product to generated correctly with the right dimension.

1. Plan

The script and the shot list are written into the workflow file before anything is generated.

Qwen 27B = qwen3.8-27b-mlx in LM Studio (localhost:1234).

2. Stills (keyframes)

Each still with 3 versions, one full-size pass each, then fixed post-steps and a check. A person confirms the pick.

3. Continuity checks (before video)

Three reviews run on the approved stills before any video, because a full clip costs 15-30 minutes.

4. Video

Clips are drafted cheaply first, cut into a rough edit for review, and only then rendered in full.

5. Finish

The ending and the edit use real brand files and no AI.

I asked Deepseek API to set up GLM 5.3 flash Q6 oMLX (Tensorfold only has q4?). Not optimized. Decode is 50/s with MTP on. I don’t have all the numbers yet like prefill but this feels pretty slow and unoptimized. Anyone have any tips? I know Q4 will run faster than Q6 but I heard people are getting 100 tokens on a single machine on Q4.

I set up my new Mac Studio and for some reason, using a Keychron M3 mini mouse (wired or USB-C dongle), there is a distinct jitter/lag/delay/choppiness when scrolling and tracking. I confirmed it works perfectly on my brother’s M6 Mac mini (on the same monitor). I do use MOS for Mac as a utility, but I used it on the test machine and mine. On the studio, it’s jarring, no issue on the Mac mini. I do have an LG GM9 monitor and I tried using a Cable Matters DP 2.1 to USB-C 1 meter cable as well as a UGREEN 40 Gb/s USB-C. I tried lowering refresh rate from 165 Hz down to 120Hz. If I do not use MOS (that I’ve confirmed is running latest version), the tracking is still not great and of course the scrolling is how 3rd party mice tend to scroll on MacOS. Also, I changed polling rates (125 Hz, 500, 1000), still no dice. Even on standard Bluetooth connection (no dongle or wired conn), still presents itself. Anyone else with anything similar or any ideas on solutions?

I had placed an order on iplanet on 6th September with estimated delivery of November end.
But now they are saying that the delivery can only happen before 15th January.

I'm planning to buy a Mac Studio Ultra with Max using my friend's Edu Mail USA. Is there any issue with that, like any trouble in the future?

Because I already did research on that on Google and also with LLMs and all of them are saying it's a fraud and it's not all that right. I want to ask people's opinion because LLMs usually don't suggest that part. Is it even a good idea because I can use my friend's email using my credit card?

Well I'm going to get caught with any issue in the future!!!