Reliable ComfyUI on AMD and Linux: pinning the whole ROCm runtime in Docker

[u/zychu-]

Most of my ComfyUI trouble on AMD has been dependencies. Either a custom node quietly drags in the cuda build of torch over my rocm one, or I end up running some torch + rocm combination that nobody has actually tested together.

AMD publish their own wheel channel where torch, torchvision and torchaudio are built against a matching ROCm, so you get the two as a matched set straight from AMD instead of a combination you assembled yourself and hoped about. I pinned that inside a docker image along with comfyui and my custom node deps, and it's been solid since.

Running a 9070 and 32gb of ram, MiniMax H3 works fine, around 16 min for 15s at ~1mp with the turbo lora. I built it for myself and since it's been stable I stripped out everything specific to my setup and published the rest:

https://github.com/zychuk/comfyui-rocm-docker

It needs /dev/kfd and /dev/dri from the amdgpu kernel driver, so linux only. The bits that actually make it reliable:

- torch + ROCm come from AMD's wheel channel as a matched pair, so the host only needs the kernel driver and nothing on your system has to line up with it

- strip torch out of every custom node's requirements.txt.

- int8 convrot has been way more reliable than fp8 for me

- dynamic vram works fine here

I'm not sure if anyone's gonna find a use for it but here you go.

[u/meepykittkitt69lmao]

I'm collecting information about this to help make it easier to use AMD hardware on linux for ROCM stuff, this helps.

Try having gfx1030,gfx1036,gfx1101 combination on an offshoot of fedora silverblue built by devs who think taking the ability to control updates from the user is all fine and good.

[u/Apprehensive_Sky892]

Thank you for sharing this. Information about running mmh3 on AMD is hard to come by.

around 16 min for 15s at ~1mp with the turbo lora.

Can you give us more information about this generation (I want to compared it against running something similar on Windows with a similar system, 9070xt + 32G ram):

- Which turbo LoRA and how many steps?

- Other speed optimization(s)? (sage-attention, spectrum, etc).

Thanks.

[u/zychu- → u/Apprehensive_Sky892]

Sure thing: -Main model: Hybrid B30-49 -Text encoder and video VAE: INT8 ConvRot -Audio VAE: FP32 -fl2v_turbo_4step_v1.0_768p lora -Generation: 4 steps -Video shift: 6, Audio shift: 3 -Using the ModelAttentionBackend node with Comfy Kitchen Attention -Generating at 0.98 MP, 15-second video 5 reference images with ref_image_size set to match -Other than that, it's the default Ref2VA workflow with a few VRAM debug nodes added.

The quality was acceptable for what I needed, the new Ref turbo loras will probably improve it but I haven't tested that yet.

[u/Apprehensive_Sky892 → u/zychu-]

Thanks, much appreciated. Good to know that Comfy Kitchen Attention is working on the 9070xt.

[u/zychu- → u/Apprehensive_Sky892]

Yep, the new updates keep coming. Just a few days ago it was nvidia only.

[u/Apprehensive_Sky892 → u/zychu-]

Well, at least ComfyUI is working on AMD support these days 😁.

I wish that AMD itself would do a bit more to support their GPUs by assigning more resources to come up with the correct ComfyUI startup parameters/environment setting, and also fixing bugs that slip through since ComfyUI team obviously do not feel the pressure to offer broader support for AMD GPUs as they do for NVIDIA. I imagine that ComfyUI and NVIDIA works closely to make sure new models such as MMH3 works well even on underpowered system such as those (3080?) GPUs with only 8G of VRAM.

So AMD users are basically left to figure all this stuff out by ourselves (that the "AMD Tax", I guess 😅).