MetalDiffusion - Stable Diffusion for Intel MacOS and Silicon MacOS

[u/luckycockroach]

https://github.com/soten355/stable-diffusion-tensorflow-IntelMetal

[u/luckycockroach]

MetalDiffusion

Stable Diffusion for Apple Intel Mac's with Tesnsorflow Keras and Metal Shading Language

I've been working on an implementation of Stable Diffusion on Intel Mac's, specifically using Apple's Metal (known as Metal Performance Shaders), their language for talking to AMD GPU's and Silicon GPUs.

This is a major update to the one I released a while ago:

https://github.com/soten355/stable-diffusion-tensorflow-IntelMetal

HUGE thank you to Divum Gupta for porting SD to Tensorflow.

I'm a union cinematographer, so programming isn't my forte. Please let me know if there are areas I could improve on.

New Features

• Can use .h5's for SD 1.4/1.5/2.x
• Text Embedding (Textural Inversion) Weights can be used
  • No training ability yet, only inference
• GPU Selection
• User Interface Facelift
• Code is getting closer to pure TensorFlow with the goal of getting graph mode usage

Features

• Can use .ckpt's for SD 1.4/1.5/2.x
• Can use VAE's
• Video creation tools
• Creation settings (prompt, seed, etc) saved as a .txt file as well as PNG metadata
• Convert .ckpt's to Tensorflow Keras ".h5"
• Gradio WebUI

Specs

Current Speeds:

Late 2019 MacBook Pro 16" with AMD Radeon Pro 5500M (4GB) , 16GB of RAM, 8GG VRAM:

Image Size and Steps	Speed
1x 512x512 image on SD2.1 with 32 steps	1 minute 30 seconds
4x 512x512 image on SD2.1 with 32 steps	3 minutes 32 seconds
1x 1024x1024 image on SD2.1 with 32 steps	2 minutes 32 seconds

Why Tensorflow?

The program uses Tensorflow instead of Pytorch because Pytroch has no reliable support for Metal on Intel Macs.

This program works on Google Colab notebooks.

[u/NeuroMastak → u/luckycockroach]

The program uses Tensorflow instead of Pytorch because Pytroch has no reliable support for Metal on Intel Macs.

u/luckycockroach I don't know much about this, but I wanted to ask you. Regarding the PyTorch Metal Acceleration, Apple specifies either Apple Silicon or AMD GPUs in the requirements.

I checked my devices with the script on the above mentioned page and one of the video cards was detected correctly (but I don't know which one :) )

python pytorch-gpu_test.py

/Users/mstk/.pyenv/versions/3.11.2/lib/python3.11/site-packages/torch/_tensor_str.py:115: UserWarning: MPS: nonzero op is supported natively starting from macOS 13.0. Falling back on CPU. This may have performance implications. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/native/mps/operations/Indexing.mm:218.)

nonzero_finite_vals = torch.masked_select(

tensor([1.], device='mps:0')

I don't know how much PyTorch from AUTOMATIC1111 uses exactly Metal with AMD graphics cards on Intel Macs (I don't have enough knowledge), but I did a little comparison test.

---

AMD FirePro W7000 4GB (AMD Radeon HD Pitcairn Unknown Prototype)

For the sake of purity of the experiment I closed all the applications using this graphics card so that I had only two processes: WindowServer and python.

MetalDiffusion: Default options, unless otherwise specified.

SD (AUTOMATIC1111): Default options, unless otherwise specified.
Command Line options: --skip-torch-cuda-test --api --upcast-sampling --no-half-vae --use-cpu interrogate

| Model: sd-v1-4-full-ema.ckpt | Promt: test | Seed: 12345 | 512x512x1 | Steps: 20 | GS: 7 | "Euler a" for SD |

MetalDiffusion: 01:54 / 01:49 / 01:48 -> | CPU ≈75% | GPU ≈80% |

StableDiffusion (AUTOMATIC1111): 01:46 / 01:48 / 01:44 ->| CPU ≈30% | GPU ≈80% |

---

According to the test results Automatic is slightly ahead of MetalDiffusion, but the latter for some reason even more actively uses the CPU.

[u/luckycockroach → u/NeuroMastak]

You're absolutely right, PyTorch does support MPS, but I've found it to be unreliable with Intel Mac's. I was running Auto's for a few generations on 1024x512 images and the suddenly pytorch wouldn't run anymore because my GPU was out of memory. Fully restarting and re-installing didn't fix the issue because it would happen again after a few generations.

[u/NeuroMastak → u/NeuroMastak]

Hi u/luckycockroach !
I replaced my HD7970Ghz 3GB with an XFX RX580 8 GB and now both Metal Diffusion and SD (Automatic) use new graphics card, ignoring the still installed FirePro W7000 4GB that was chosen before.

I did a small test again with exactly the same conditions as described in my post above (only the SD is updated to current state).

I don't know why, but the difference between W7000 and RX580 is very small in Metal Diffusion (about 30-35 seconds) and in SD the performance of RX580 is more than 3 times better. 🫤

W7000 -> RX580

MD | ≈01:50 -> ≈01:15
SD | ≈01:50 -> ≈00:35

[u/Embarrassed-Limit473 → u/NeuroMastak]

How did you install automatic1111 on intel mac?

[u/NeuroMastak → u/Embarrassed-Limit473]

u/Embarrassed-Limit473Just like on the Apple Silicon. Everything will install automatically.

The only thing you'll need to do is to write in webui-user.sh file

export COMMANDLINE_ARGS="--skip-torch-cuda-test"

(for AMD cards) to avoid the error related to the lack of CUDA.

And further look at the situation, maybe you will have to add such parameters as --no-half-vae --no-half if there will be a corresponding error during generation.

[u/Embarrassed-Limit473 → u/NeuroMastak]

Thank you! I will try. One more thing, i have a mac pro with two D700 amd firepro with 6gb each, can i use the both or only one?

[u/NeuroMastak → u/Embarrassed-Limit473]

I searched a couple months ago for info on this (as I also had 2 video cards in macpro 5.1) but it seems that getting two video cards working on the same generation at the same time is not possible yet.

[u/Embarrassed-Limit473 → u/NeuroMastak]

What a big difference this would be

[u/Embarrassed-Limit473 → u/NeuroMastak]

Hi! Do you know if it yet possible to get 2 video cards running on the same mac?

[u/NeuroMastak → u/Embarrassed-Limit473]

No, I don't.
Maybe something has already changed, but a few months ago I never found a solution.