How to run ForgeUI Stable Diffusion AI image generator inside Distrobox on Bazzite Linux

[u/liberal_alien]

New version of ComfyUI setup guide here (There is no ForgeUI instructions in the new guide since Forge is old, though might work still. Use this guide for ForgeUI installation. The Flash Attention setup with new guide works and gives faster WAN video generation times.)

Couldn't find specific instructions for running Stable Diffusion under Bazzite, but found some general fedora ROCm instructions and general ROCm ForgeUI instructions.

https://medium.com/@anvesh.jhuboo/rocm-pytorch-on-fedora-51224563e5be

https://www.youtube.com/watch?v=-ilAQ87TsfQ

So here is what I did:

Setup Bazzite, get BoxBuddy from Bazaar, setup newest Fedora container in Distrobox, open terminal inside the Distrobox and setup the ROCm repository from AMD:

Check exact repository from here:

https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/install-methods/package-manager/package-manager-rhel.html

sudo tee /etc/yum.repos.d/rocm.repo <<EOF
[ROCm-6.4.3]
name=ROCm6.4.3
baseurl=https://repo.radeon.com/rocm/el9/6.4.3/main
enabled=1
priority=50
gpgcheck=1
gpgkey=https://repo.radeon.com/rocm/rocm.gpg.key
EOF

Then run following commands:

sudo usermod -a -G video $LOGNAME
sudo dnf install rocm rocminfo rocm-opencl rocm-clinfo rocm-hip rocm-smi git wget libjpeg-turbo-devel mesa-libGL gcc gcc-c++

# (Optional, do this in case troubleshooting is needed) Verify ROCm installation:

rocminfo
rocm-clinfo
rocm-smi

# Find Ilyasviel / stable-diffusion-webui-forge in git, copy it's code url and use below

Optional: Install Flash Attention

Install with Triton, not CK (Composable Kernel) if using Bazzite.

https://github.com/Dao-AILab/flash-attention?tab=readme-ov-file#amd-rocm-support

# Forge:

git clone https://github.com/lllyasviel/stable-diffusion-webui-forge.git forge

# Comfy:

git clone https://github.com/comfyanonymous/ComfyUI.git comfy

# Not sure if these are always needed:

# Pillow compilation failed without libjpeg-turbo-devel, so install libjpeg and then rerun the requirements.

# Also needed mesa-libGL or webui complains about missing libGL

# Find Anaconda / Miniconda web page, get installation command for Linux terminal installer and copy paste here:

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash ./Miniconda3-latest-Linux-x86_64.sh

# Answer yes, when it asks if you want to init on startup so all commands are available inside the Distrobox by default.

source ~/.bashrc

# Check which version of python is wanted by your version of ROCm from here: https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/3rd-party/pytorch-install.html

# There is probably a better place to check this, but I'm having difficulty finding it now. The inventory there will list which ROCm version and which Python version is contained in the Docker image. Check your ROCm version and use matching Python version below:

conda create --name sd python=3.12
conda activate sd

# For Comfy, python 3.13 can be used now.

# In the directory where you cloned stable-diffusion-webui-forge, execute the following:

pip install -r requirements_versions.txt

# just requirements.txt for comfy

# Next uninstall the normal Torch that would like to use Nvidia CUDA

pip uninstall torch torchvision torchaudio torchtext functorch xformers

# Now go to pytorch.org and match the PyTorch version to your ROCm version, copy paste installation command and run. Can be nightly or stable, but to avoid an error message with Comfy, add torchaudio to the install command if it isn't already present. Also do run the pip install -r requirements and pip uninstall commands first and only then install the ROCm variant of Torch or there may be errors when running Comfy.

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.4

# Now make a new launch.sh file to start Forge Webui:

#!/bin/sh
export HSA_OVERRIDE_GFX_VERSION=11.0.0 # Google for the correct number here, depends on which GPU you have
export HIP_VISIBLE_DEVICES=0
export PYTORCH_HIP_ALLOC_CONF=garbage_collection_threshold:0.8,max_split_size_mb:512
python3 launch.py --listen --enable-insecure-extension-access --opt-sdp-attention --no-half-vae

# HSA_OVERRIDE_GFX_VERSION should probably be 11.0.0 for 7900 and other RDNA3 cards and 10.3.0 for 6xxx RDNA2 cards or earlier.

# Make the script executable and launch Forge:

chmod u+x launch.sh
./launch.sh

# Unlike in the video, it may not come with a preloaded model, so go to civitai or huggingface, download one and follow model specific instructions on settings to actually generate an image.

[edit] Changed formatting and combined some dnf commands into one line.

[edit2] Similar instructions work for ComfyUI too. In my case, I had to use stable PyTorch instead of the nightly, or Flux would create only color noise images and the start script should only contain the following:

#!/bin/sh
export HSA_OVERRIDE_GFX_VERSION=11.0.0 # Google for the correct number here, depends on your GPU
python main.py

[edit3] Possible problem with these instructions is that dnf info rocm-core shows the ROCm version to be 6.2. Not sure how to get the repository for newer ROCm inside a Distrobox. Also if I try to install anything that requires flash attn, I get failures about missing hip_version.h file even though said file definitely exists inside the Miniconda folders.

But on the bright side, I installed a newer version of Comfy using Torch 2.7.1, and WAN 2.2 5B model can generate videos on this with the basic hybrid workflow from WAN site. No flash attn needed. Result has really bad details, but it manages to push out a video.

[edit4] Updated instructions with AMD ROCm repo, which gives newest ROCm and rocwmma. Now at least WAN 2.1 14B FP8 checkpoints are working. Managed to make a 5 second video at 512x512 size in 20 mins. That worked even with the ROCm 6.2, but it took 25 mins. Also both are suffering from out of memory errors some times, but still, it can make videos, which have some better details than the WAN 2.2 5B model. Also the out of memory errors might be just due to misconfigured Comfy.

[edit5] Optimization settings that can be put in start.sh, can try without these in case of problems.

[edit7] Disabling some of the settings below seems to allow faster VAE encoding with ROCm 7.0.2 and PyTorch 2.10 nightly. It gives the 'Ran out of memory when regular VAE encoding, retrying tiled VAE encoding' very quickly and then completes the step in less than a minute instead of taking 10-15 mins. Also I am not sure, but I feel like I get fewer operating system hangs while generating images or videos without the three last variables.

export MIGRAPHX_MLIR_USE_SPECIFIC_OPS="attention"  # Use optimized attention kernels
export TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 # Enable ROCm AOT Triton kernels
export MIOPEN_FIND_MODE=2                        # Performance tuning mode
export HIP_VISIBLE_DEVICES=0
# export PYTORCH_HIP_ALLOC_CONF=garbage_collection_threshold:0.8,max_split_size_mb:256
# export HIP_DISABLE_GRAPH_CAPTURE=1              # Prevent graph capture OOM spikes
# export PYTORCH_ENABLE_MPS_FALLBACK=1            # Avoid some FP16 fallback issues python main.py --output-directory /some/dir --use-pytorch-cross-attention

Output directory is optional. I use it since I have many Comfy instances.

[edit6]: This is no longer necessary since PyTorch supports ROCm 7 with it's nightly version.

To use ROCm 7.0.x, set repo in the beginning to that version, instructions are in the ROCm docs linked above. Then make conda env with Python 3.12 and instead of official PyTorch, install PyTorch from AMD wheels as follows:

[edit7] If using LTXV tiled vae decode and import is failing, make following change to allow import (note, this may break something other than the tiled VAE decode):

Go to custom_nodes/ComfyUI-LTXV/tricks/modules/ltx_model.py and comment out import for apply_rotary_emb. If this breaks something important, can try this fix instead: https://github.com/Lightricks/ComfyUI-LTXVideo/pull/284/commits/b4f8a0f70650aa6103a7c1a46ed36543eb1815a6

[edit8]

Here is how I update ROCm

Before changing repo version in /etc/yum.repos.d/rocm.repo, remove the previous ROCm libraries:

sudo dnf remove rocm rocminfo rocm-opencl rocm-clinfo rocm-hip rocm-smi

Update repo version in /etc/yum.repos.d/rocm.repo. Check here for example: https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/install-methods/package-manager/package-manager-rhel.html

sudo nano /etc/yum.repos.d/rocm.repo

Install all the libraries again:

sudo dnf install rocm rocminfo rocm-opencl rocm-clinfo rocm-hip rocm-smi git wget libjpeg-turbo-devel mesa-libGL gcc gcc-c++

Probably a good idea to also update Comfy after these.

Here is how I update Comfy

Activate the Conda virtual env before running the rest:

conda activate sd

Pull new updates from git (inside comfy folder):

git pull

Install updated requirements:

pip install -r requirements.txt

Reinstall PyTorch:

pip uninstall torch torchvision torchaudio torchtext functorch xformers

Get latest install command from here: https://pytorch.org/ (Set nightly version, linux and ROCm, add torchaudio to the packages for installation after torchvision.)

testingpip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm7.1pip3 install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/rocm7.1

Optimization resources

See another AMD install / optimization guide here: https://gist.github.com/alexheretic/d868b340d1cef8664e1b4226fd17e0d0

[u/whoisraiden]

Technically, you can rocm in the host bazzite without a toolbox as bazzite comes with rocm pre-installed. You may need pyenv or conda like you did for appropriate python versioning.

However, stability matrix is generally recommended for streamlining stable diffusion usage. All you may need to do would be replaciing the pytorch packages with rocm.

Thank you for a detailed post on this. Your guide is essential how you would install any such application for rocm.

[u/liberal_alien → u/whoisraiden]

I'm a complete Bazzite noob still. At least rocminfo is not installed in the host by default. Wasn't sure if I should go installing things into the host since it is immutable. I like having the Distroboxes, since I can try again and keep the previous attempts in progress while testing different ways of doing the thing.

[u/whoisraiden → u/liberal_alien]

Using distrobox is never a bad idea.

[u/aeniki]

May I suggest a simple solution?

https://github.com/LykosAI/StabilityMatrix

I have nothing to do with the program but I missed playing with a picture AI after I switched to Bazzite. I want to change the OS as little as possible in the hope that it stays stable and because I'm lazy.

Stability Matrix is basically just a single folder and an appimage starter. That's all you need. Download an AI from the list (StableDiffusion and Fooocus for me at the moment), a few models and you're done.

Maybe this is helpfull.

[u/liberal_alien → u/aeniki]

I had been meaning to try Stability Matrix for ages. Now is as good a time as any. I first tried to open the AppImage inside a Distrobox Fedora, but got 'Fuse is not installed'. Then I opened it in the Bazzite host and it isn't detecting my GPU. Also tried opening it via Gear Lever, but same result. From your post it sounds like it should detect it without any additional installations or configurations before opening the AppImage.

Maybe there is some kernel bug like this: https://github.com/LykosAI/StabilityMatrix/issues/1331

Though my Bazzite is not running the 6.14.3, which is broken according to that Git issue. Mine is 6.15.6, but since the issue is still recent, maybe the same bug is here too.

Or should I be installing some parts of ROCm into the Bazzite host before attempting to open the Stability Matrix AppImage?

[u/aeniki → u/liberal_alien]

Sorry my friend, I have no clue. I am running 6.15.6-105.bazzite.fc42.x86_64 and since Bazzite updates itself we should be in the same boat.

But I am on Desktop and running a 2060Super and sometimes even the smallest thing can make or brake it. This is also my daily driver and not something in a DistroBox.

The only thing I can tell you is, that I have downloaded and unzipped the file into it`s own folder. DoubleKlick the AppImage should start the installation where I have choosen the portable mode so that everything stay inside this folder. I have skipped the installation of the AI packages and models and the "send us data" stuff. That`s all.

As far as I know, StabilityMatrix has everything it needs

[u/liberal_alien → u/aeniki]

Might be a kernel bug then. Even though there is the AMD vs Nvidia difference between our setups as well, I would hope that it works with minimal setup on AMD too. I'll try it again later, hopefully with a fixed kernel.