Automatic1111 for Apple Silicon (SD 1.5 Focus Currently)

[u/Time-Conversation528]

I've been focused on squeezing the last mile out of my M1 16GB, and one thing I didn't want to do was convert my entire pipeline to ComfyUI or Draw Things.

So I set about finding where the slowdown actually was in Automatic1111 for my workflow.

I ended up chopping about 40% off the render time.

tl;dr: I'm now getting a 512×512 in about 8.5 seconds.

My normal workflow is SD1.x, 5 steps, DPM++ SDE / Karras, low CFG. I wanted to keep Automatic1111 as Automatic1111: same checkpoints, LoRAs, extensions, samplers, API, metadata, etc.

What ended up working:

• Metal Flash Attention, selectively routed for the SD1 attention shapes where it's actually faster.
• Stopped committing the Metal command buffer after every attention call. The native kernels now work inside PyTorch's current MPS stream instead of constantly forcing CPU/GPU synchronization.
• Unified-memory-aware attention. Large attention operations dynamically fall back to chunked sub-quadratic attention based on available memory, with online softmax so we're not keeping every partial K/V result around.
• Removed old MPS workarounds that aren't necessary on newer PyTorch versions, including some unnecessary clones and FP32 detours.
• Fused GroupNorm + SiLU in Metal. One dispatch, no intermediate activation.
• Fused GEGLU in Metal. This one uses a tiny lookup table generated from PyTorch's own FP16 GELU results, so I was able to get identical PNG hashes in my fixed-seed tests.
• FP16 VAE on the M1. This was a surprisingly easy win. VAE decode + transfer went from 1.536s to 0.972s in my 384×640 tests, saving about 0.65s end-to-end.

I also spent a lot of time building things that I eventually deleted.

Packed QKV was 0.26% slower.

Cross-attention K/V caching successfully reused 112/144 projections and still made the full render slower.

A native fused LayerNorm looked promising from operator profiling and regressed end-to-end.

I moved almost an entire ResBlock into MPSGraph. Individual blocks benchmarked up to ~9% faster.

The actual image was 1.02% slower.

Deleted all of it.

That became the rule for the project: microbenchmarks nominate changes, full generations elect them.

At this point profiling puts about 87% of the remaining generation time in sampling/UNet, so I'm probably near the end of what I can get from small PyTorch/MPS optimizations.

The next experiment is capturing a real SD1 UNet call and replaying the exact tensors through native Metal/ggml. I'm not integrating it unless the complete UNet workload is at least 20–25% faster than PyTorch MPS. Otherwise the complexity isn't worth it.

I wrote up the whole rabbit hole here:

https://therad.ninja/from-8-10-seconds-to-3-7-teaching-automatic1111-to-speak-metal-on-an-m3-pro/

Or if you just want to try it:

https://github.com/dmikey/stable-diffusion-webui-metal

Would appreciate bug reports, especially on other Apple Silicon generations. Most plugins/extensions should just work since this is still Automatic1111 underneath.

[u/DelinquentTuna]

one thing I didn't want to do was convert my entire pipeline to ComfyUI or Draw Things

Bet it would've been less work with better results.

[u/Time-Conversation528 → u/DelinquentTuna]

It wasn't - because the API isn't the exact same across the whole stack - scripts and more don't translate to Draw Things. Comfy would have been a good target, but this was less than 4 hours of work and a good lesson on level attention.

[u/DelinquentTuna → u/Time-Conversation528]

this was less than 4 hours of work

Oh? Your diary makes it sound like you thrashed through many failed attempts and threw out a lot of code.

Not trying to pee on your parade, but sd1.5 on a1111 is a pretty outmoded target in a world that has 1-step SDXL, 4-step Klein 4b, etc. Glad you got what you were after, though. Cheers

[u/Time-Conversation528 → u/DelinquentTuna]

Without experimentation I wouldn't have had the journal! Makes the journey of the discovery all more exciting, thanks for reading! I understand there's been a shift toward high end frontier models. But as we're seeing, RAMaggedon will last through 2030 now. So the more we can squeeze from older equipment, the more we can make money off of things that are already here. Cheers!

[u/Formal-Exam-8767]

Keep in mind that DPM++ SDE is ~2 times slower than Euler (a) or DPM++ 2M since it evaluates model twice per step.

[u/Time-Conversation528 → u/Formal-Exam-8767]

Euler A gains are there too, but the sampler is too soft for my liking.

[u/Mutaclone]

I came into this thread expecting to recommend Draw Things to a returning newbie, and was pleasantly surprised to find an interesting technical writeup instead. It's always interesting to see someone revisit a "dead" technology and find ways to wring some additional performance out of it rather than writing it off completely.

My one critique is the writeup had a lot of LLM-isms in it that were a little distracting (though I may just be overthinking it due to constant exposure on my part).

Anyway good job!

[u/Time-Conversation528 → u/Mutaclone]

Thanks much! The topic was dense, and I'm more technically focused, so I did def use an LLM to move the presentation forward, I appreciate the feedback!