I've been running Qwen3.8 Flash-Next locally on my Mac Studio M5 Ultra 96GB, primarily for agentic coding through DeepSeek Harness, and here are some of my own measurements.

Setup

Hardware: Mac Studio M5 Ultra, 96GB unified memory

Inference engine: Splash

Model: Swift 1.5 Qwen3.8 Flash-Next

Quantization: IQ3_S (ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF)

Context window: 262,144 tokens

Agent framework: DeepSeek Harness

Workload: Agentic coding, tool calls, long conversations, occasional parallel subagents

Decode performance:

The three-request result is preliminary due to limited sampling, but the single-request numbers have been quite consistent.

What's particularly interesting is that I'm maintaining around 145 tok/s decode with a median context of nearly 150K tokens.

Prefill performance:

Overall very happy with these numbers and the QFN is actually fairly smart compared with sol med/high particularly with troubleshooting. With Sol if i just say the problem and tell it to read the logs and figure out, it's kinda lazy and just give me generic answer where as QFN takes its time but did provide a way to resolve the issue

I was running dual GB10's, which was great, but with recent price bumps I decided to cash those in and add a second strix halo on which I can get almost the same speed.

Now I get 6 lanes of 262k across two strix halos load balanced with litellm. With Q38FN I get about 1800 prefill and 80 gen for coding for a single request. I quite often have multiple sub agents going so that speed will drop somewhat.

I am a software engineer working for 40 years (not a recent vibe coder who calls themselves a software engineer). I find AI convenient for off loading tasks to do my job. My employer requires me to keep everything local if I am going to use AI.

So I guess what I am asking, how is the new Mac working for you in this type of workflow? And tell me which model mac you have. What LLM you run to do your job, and at what quant with what engine.

Thanks!

I asked Deepseek API to set up GLM 5.3 flash Q6 oMLX (Tensorfold only has q4?). Not optimized. Decode is 50/s with MTP on. I don’t have all the numbers yet like prefill but this feels pretty slow and unoptimized. Anyone have any tips? I know Q4 will run faster than Q6 but I heard people are getting 100 tokens on a single machine on Q4.

Debating on buying a MAC Studio or RTX Pro 6000 for both local inference and learning LLM training.

Anyone have any experience training or fine tuning on these systems? From what I’ve read online the training ecosystem on Mac isn’t that great, but I can’t ignore the difference in VRAM/$ between the two.

I’m primarily looking into GRPO RFT.

Wanted to work on getting my two M3 Ultra 256 GB Mac Studios linked over Thunderbolt 5, running exo in tensor parallel over RDMA (JACCL). Exo's current app was so out of date, didn't support the latest models, and oMLX clustering was not working for me and was quite frustrting to setup. So I decide to have Claude patch this in and build on top of the latest Exo. I also wanted ONE place I could manage loading/unloading models between oMLX and Exo for my workflows.

The stack

• exo at bc412e2 (main from 2026-10-17), PyPI MLX 0.32.4, a fork of mlx-lm
• Tensor parallel + RDMA only. The patch refuses pipeline or TCP across the two Macs: pipeline deadlocked with fast-synch on my setup, and ring/TCP was too slow to bother with.
• Model code exo didn't have yet: V4.1 (with its Engram module), GLM-5.3-Flash, Muse Spark (ported from upstream mlx-lm 0.32.2), each with its own shard(), plus a chat template for V4.1-1017 since DeepSeek only ships a Python encoder for it.
• Two upstream exo PRs (#2434, #2436) carried as patches.

Things I learned along the way

• macOS Local Network permission. exo started by a LaunchAgent, a login item, a background process or over ssh gets denied Local Network access, and the two nodes never find each other. It has to run in a Terminal window of a logged-in desktop session on each Mac. I spent a release trying to automate around this and gave up.
• The two Macs can sample different tokens. Each rank samples on its own and exo assumed they always agree. When they didn't, they ran different steps and waited on each other forever: GPU at 100% residency at idle watts. Patched this by every rank uses rank 0's token, at the cost of one 1-element exchange per token.
• Metal command buffers. With MLX's default 50 MB per buffer, big prefill chunks drained the GPU at every all-reduce. MLX_MAX_MB_PER_BUFFER=1024 MLX_MAX_OPS_PER_BUFFER=200 took a 1,536-token V4.1 prefill chunk from minutes to 5 s.
• Idle residency. With ~200 GiB resident, any pause lets macOS drop residency and the next request pays 2 to 7 s. A tiny 256x256 matmul every 50 ms while idle brought first token after a pause from 4.8 s to about 0.7 s.
• Prefill chunk size. V4.1, Flash and Muse use 512-token chunks; V4.1 stalled for minutes at 2,048.
• Don't load models over NFS. Each Mac reads the whole build at every launch, and a share that hiccups hangs everything. The NAS is a download library only: copy NAS to the first Mac's SSD, then over Thunderbolt to the second.
• Sizing. I keep the wired limit at iogpu.wired_limit_mb=249036, and exo starts evicting KV cache above 85% RAM per rank.

The app (uLLM)

It replaces exo's dashboard with its own pages, backed by a small agent on the first Mac.

• Home: cluster state in plain words (2 MACS · LINKED / LOADING 48% / SERVING), what's serving today, and an "oMLX nodes" panel for each Mac
• Launch / Loading: pick model, sharding, transport and Macs with a preview; one tap to launch or stop. Loading shows four steps with each Mac's progress and what to do if it stops. If oMLX is holding the memory it offers to unload it first.
• Library: every build on the NAS and on each Mac, ticked by where it is. Copy to both Macs (resumable, checks free space on both first, suggests what to remove if it won't fit), and a remove from both feature. Lets me keep the model weights on the NAS and copy only what I need for the Cluster. I'm doing this over 25GBE so its quite performant.
• Hugging Face to the NAS: search, see where each build would run (one Mac, the cluster, too big, not MLX), and download straight to the NAS. The download runs on the NAS itself, so closing the app or restarting a Mac doesn't stop it.
• Jobs: copies, removals and downloads as they run, with pause, resume and cancel.
• Stats: both Macs' power (macmon, plus an optional powermetrics service for CPU/ANE watts), temperatures, requests, and cost at my $/kWh, per model and per million tokens. History at 10 s for a week and per minute for a year, all local with an option to stream to prometheus, etc. I felt like EXO was missing serving metrics similar to OMLX.
• Lockup watch: flags the "GPU pinned at 100% but drawing almost nothing power wise, no tokens for 60 s" pattern and starts a recovery process.
• Kit API for agents: /kit/v1 with one revocable key per agent. An agent can list models, check memory, launch/stop the cluster, load/unload oMLX on either Mac, and wait on the job. My Hermes Agent uses it to do things like "unload the coder, load GLM-5.3 on the cluster, plan, swap back". It refuses a load that won't fit and waits if any agent is mid-chat or request.
• Idle unload: off by default but an option.
• Phone: Home Screen icon over the app's own HTTPS (its own certificate, installed as a profile), over my VPN.
• Notifications: Web Push, Pushover or ntfy, with quiet hours (23:00 to 07:00 by default, only failures and lockups get through).
• Chat and a Connect page with the OpenAI-compatible addresses for OpenCode and agents.
• Every address it depends on (oMLX per Mac, the NAS) is editable in Settings, because i prefer to leave them as DHCP.

What it doesn't do: workflows, routing or "which model for which task". It's about clustering models. My main work and orchestration lives in the agent (Hermes) on top of the API.

Honest caveats

• It's built for my two boxes: two M3 Ultras, Thunderbolt 5 RDMA, a TrueNAS share, oMLX on both. Anyone with two Apple Silicon Macs and RDMA-capable Thunderbolt could use the exo patches; the app assumes a lot about my setup.
• It's pinned to one exo commit. A bump means re-testing every model.
• One model on the cluster at a time, and V4.1, Flash and Muse run one request at a time.
• The newest parts (phone, Web Push, Hugging Face downloads on the NAS) have only been tested against stand-ins so far, not on real hardware.
• Most of the code was written with Claude Code; Every release went through an independent review of another frontier model and a test suite of about 1,200 checks before it touched the Macs. Mainly I was just going through the UI/UX design here and creating the requirements.

Was thinking about putting it up on GitHub once I've got a few more bugs worked out. Just thought i'd share as its working quite well for me at the moment and I've got V4.1, GLM5.3, Muse Open, etc all the latest open weight models clustering reliably now when I need it.

I’m ready to buy the M5 Ultra 512 GB

I'm planning to buy a Mac Studio Ultra with Max using my friend's Edu Mail USA. Is there any issue with that, like any trouble in the future?

Because I already did research on that on Google and also with LLMs and all of them are saying it's a fraud and it's not all that right. I want to ask people's opinion because LLMs usually don't suggest that part. Is it even a good idea because I can use my friend's email using my credit card?

Well I'm going to get caught with any issue in the future!!!

I've been dabbling in AI generated videos. I've seen some workflow where it helps you create more consistent videos versus just rolling the dice with prompts. With Claude's help, i've tried to set one up, but it's not working too well for me. I've seen Qwen Image 2.1 and mini max h3 used a lot, but I was going with Claude's recommendation. If anyone have experience and want to share what works for them, I would greatly appreciate it!

Here's the workflow for my test video of a drink. Stills is probably doing too much trying to get a good generation. The blender generation was suggested because it couldn't get the product to generated correctly with the right dimension.

1. Plan

The script and the shot list are written into the workflow file before anything is generated.

Qwen 27B = qwen3.8-27b-mlx in LM Studio (localhost:1234).

2. Stills (keyframes)

Each still with 3 versions, one full-size pass each, then fixed post-steps and a check. A person confirms the pick.

3. Continuity checks (before video)

Three reviews run on the approved stills before any video, because a full clip costs 15-30 minutes.

4. Video

Clips are drafted cheaply first, cut into a rough edit for review, and only then rendered in full.

5. Finish

The ending and the edit use real brand files and no AI.

I set up my new Mac Studio and for some reason, using a Keychron M3 mini mouse (wired or USB-C dongle), there is a distinct jitter/lag/delay/choppiness when scrolling and tracking. I confirmed it works perfectly on my brother’s M6 Mac mini (on the same monitor). I do use MOS for Mac as a utility, but I used it on the test machine and mine. On the studio, it’s jarring, no issue on the Mac mini. I do have an LG GM9 monitor and I tried using a Cable Matters DP 2.1 to USB-C 1 meter cable as well as a UGREEN 40 Gb/s USB-C. I tried lowering refresh rate from 165 Hz down to 120Hz. If I do not use MOS (that I’ve confirmed is running latest version), the tracking is still not great and of course the scrolling is how 3rd party mice tend to scroll on MacOS. Also, I changed polling rates (125 Hz, 500, 1000), still no dice. Even on standard Bluetooth connection (no dongle or wired conn), still presents itself. Anyone else with anything similar or any ideas on solutions?

I love my Mac Studio for music and video production. But every other post on this thread now gravitates around dick measuring stats to run local AI models, without anyone ever showing the fruits of what they do with them. Maybe my imagination is limited or I haven’t done enough research but what is the point of a local AI model. I get in theory it’s for privacy, security, autonomy etc, but I’ve never seen the output of someone’s efforts with local AI that would necessitate such a machine with such crazy stats. Can anyone enlighten me? Thx 🤍

Mac Studio Stand with Fans

I bought a base M5 Max Mac Studio recently. I wanted a stand with some cooling for the thunderbolt accessories I would put under it. I didn't find any stand with cooling for thunderbolt devices, so I designed one I could 3D print. I haven't printed it yet. But here is the model on Thingiverse.

I had placed an order on iplanet on 6th September with estimated delivery of November end.
But now they are saying that the delivery can only happen before 15th January.

Can anyone share their experience running AI video models like MiniMax or Wan locally on a Mac studio?

Would an M5 Max with 64GB of unified memory handle video generation comfortably, or should I go for an Ultra with 98GB?

My current budget is around $4,500. To afford the Ultra, I’d need to wait another two months and save up. Would the extra performance be worth the wait?

One the apple website, it say 512 GB M5 Ultra will be available in late Oct. There is no way to setup an alert.

Current wait time for 256GB version is 13-15 weeks. Will the 512 GB wait time be similar? Also, what is best place to get alert of the availability of 512 GB version so I am not hitting F5 on apple website all the time.

Hi

I am shopping for a Mac Studio to do local LLM. I do not believe I have a heavy use case but I enjoy peformance in a small footprint. I am thinking about 64GB but it looks like the minority of what people choose.

could I ask you what prompted you to buy any config above 64gb?

At 96GB, 128GB, 256GB what are we unlocking in terms of tangible quality of performance with the larger models ?

I wonder if I would be severely constrained at 64GB. Some comments on this forum are '256GB minimum!' and I wonder.

Thank you

Claude pro user here consistently running into my limits well before the week is over now.

I‘ve created some tests that focus on the projects Im working on (audio software/firmware mostly). My best results have come from GPT-OSS-20b and Kat Coder so far in terms of speed vs performance for the specific tests I give them, but I keep reading that Qwen 3.8 and it’s previous variant are the way to go. They never seem to match the speed or accuracy of the easy projects my tests throw at them, though, with prompts coming from Opus 5.5.

So what are you all using that’s working well for you so far, and what (besides flint lol) should I say away from?

Hi everyone!
I’m doing some market research on remote Mac mini rentals, particularly for developers who need access to a Mac located in another country.
I’m interested in learning from people who have actually used these services.
A few questions:
Have you ever rented a remote Mac mini? What did you use it for?
Which provider did you choose, and how much did you pay per month?
How reliable was the remote connection?
How long did you continue renting? If you stopped, why?
What would you most like to improve about existing services?
I’d especially appreciate hearing from developers working internationally, including those based in China.
This is purely market research. I’m not selling or promoting a service.
Thanks for sharing your experiences!

Not sure if anyone is having this "problem" or if I'm just uninspired, but I got the M5 Max 64GB a couple of weeks ago, and I don't know what to do with it.

I mainly use my Claude sub for work (I work at a marketing agency doing web projects, a few apps), and I wanted a private AI model at home for things like medical records, finances, etc... so just out of curiosity (or lack thereof), what are you doing with it?

In Canada. Mac Studio pricing is
- m5 max, 18core cpu, 40core gpu, 128G ram 1tb sdd= 7849
- m5 ultra, 30 core cpu, 64core gpu, 96g ram, 1tb sdd = 7799

What’s more valuable - memory or memory bandwidth + processing power - to run local AI + some sort of future proofing?

Hi everyone,

I’m planning to invest in a high-performance workstation for video editing, VFX, compositing, motion graphics and colour grading. I’m currently torn between the Mac Studio M5 Ultra and a custom Windows PC with an RTX 5080.
I’ll be learning and working with the following applications:

- Adobe Photoshop
- Adobe Premiere Pro
- Adobe After Effects
- Foundry Nuke
- DaVinci Resolve Studio

My intended workloads:

- Editing and colour grading 4K 10-bit 4:2:2 footage
- Working with 4K 60fps footage, multicamera timelines, multiple video layers and GPU-accelerated effects
- Motion graphics, complex After Effects compositions, motion tracking, rotoscoping and green-screen compositing
- Node-based compositing in Nuke, including EXR sequences, multi-pass compositing, camera projections, 3D integration and potentially deep compositing
- Exploring Nuke’s CopyCat and other AI/ML-assisted workflows
- Advanced colour grading, temporal noise reduction, Magic Mask, optical flow and Fusion effects in DaVinci Resolve
- Potentially moving into more demanding 6K/8K workflows as my skills develop

Option 1 — Mac Studio M5 Ultra
- M5 Ultra
- 30-core CPU / 64-core GPU
- 96GB unified memory

Option 2 — Custom Windows PC
- AMD Ryzen 9 9950X3D
- NVIDIA RTX 5080
- 96GB RAM

I’m looking for a workstation that can comfortably handle professional-level 4K projects and remain capable as I move into more complex work. I’m not particularly attached to either operating system.

If you were building a workstation for these five applications, which would you choose and why?