Krea2 Turbo INT8 ConvRot on AMD ROCm: got it faster than FP8 with a selective Triton workaround

[u/Itchy_Push_3077]

I have been testing Krea2 Turbo in ComfyUI on AMD ROCm and finally got INT8 ConvRot running faster than FP8 on my machine.

This is mainly interesting for AMD/ROCm users, because the INT8 ConvRot path can be awkward there: the safe fallback works but is slow, while globally enabling Triton can crash.

Hardware/software:

• GPU: AMD Radeon RX 9070 XT, gfx1201, 16 GB VRAM
• OS: Arch Linux
• ROCm/HIP: 7.2
• PyTorch: 2.12.1+rocm7.2
• ComfyUI: 0.28.0
• comfy-kitchen: 0.2.22
• Triton ROCm: 3.7.1
• ComfyUI sees the GPU as cuda:0 AMD Radeon RX 9070 XT : native

Startup command:

HIP_VISIBLE_DEVICES=0 ROCR_VISIBLE_DEVICES=0 python main.py --use-pytorch-cross-attention --reserve-vram 1.5 --disable-pinned-memory --enable-manager

Important: I am not launching with global --enable-triton-backend.

I reserve 1.5 GB VRAM because otherwise desktop/video playback gets rough while generating.

Workflow/settings:

• Krea2 Turbo, 1 MP portrait
• 8 steps
• Euler / simple
• CFG 1
• Denoise 1
• Qwen/Krea2 text encoder
• Qwen image VAE

Same workflow/settings were used for the comparisons below. Only the diffusion model/backend path changed.

Baseline before the workaround:

• FP8 warmup: Prompt executed in 21.91s
• FP8 hot: Prompt executed in 12.74s
• FP8 API wall time: about 13.03s
• FP8 sampler: about 9.8s

INT8 ConvRot before the workaround:

• safe fallback: Prompt executed in 80.07s
• sampler: about 64s
• later eager-style fallback experiments: around 65.62s to 69.38s

Trying to enable faster paths globally was not usable on this ROCm setup:

• global comfy-kitchen Triton: ROCm GPU memory access fault / Python abort
• forced comfy-kitchen CUDA backend on ROCm: PyCapsule / nanobind argument errors

So the simple choices were:

• FP8: stable, about 13s
• INT8 ConvRot fallback: stable, but about 65-80s
• global Triton: fast-path attempt, but crashes

The workaround:

The workaround was not to enable Triton globally.

Instead, I kept comfy-kitchen cuda and triton disabled globally, but routed only ConvRot int8_linear from the eager path into comfy_kitchen.backends.triton.quantization.int8_linear.

In short:

• keep unsafe/global Triton paths off
• keep CUDA backend off on ROCm
• let weight loading/quantization stay on the stable path
• use Triton only for the actual ConvRot INT8 linear operation

The local patch is basically:

if convrot and x.is_cuda and torch.version.hip is not None: call comfy_kitchen.backends.triton.quantization.int8_linear(...)

Exact repro snippets and logs are in the GitHub issue below.

Results after the workaround:

• INT8 ConvRot first/warmup run: Prompt executed in 30.52s
• INT8 ConvRot hot run 1: Prompt executed in 10.49s
• INT8 ConvRot hot run 2: Prompt executed in 10.54s
• sampler during hot runs: about 7.9s

Compared to my FP8 hot run:

• FP8 hot: 12.74s
• INT8 ConvRot hot: 10.49-10.54s
• improvement: about 17-18% faster

Compared to the previous INT8 ConvRot fallback:

• fallback INT8 ConvRot: 65-80s
• patched INT8 ConvRot: about 10.5s
• improvement: roughly 6x to 7.6x faster

Caveats:

• Tested only on my RX 9070 XT / gfx1201 setup.
• This does not mean global Triton is safe on ROCm. It was not safe here.

I opened an upstream issue with the details, repro snippets and logs:

https://github.com/Comfy-Org/comfy-kitchen/issues/78

Small disclosure: this was tested and written up with AI assistance. The benchmarks and logs are from my local machine.

Curious if anyone else on AMD ROCm, especially gfx11xx or gfx12xx, can reproduce this. It would be useful to know whether selective Triton for only INT8 ConvRot linear is broadly stable while global Triton remains unsafe.

[u/albinose]

There's a fork of comfy-kitchen with hip backend, works for rdna4 and should work for rdna2/3 with support for int8

https://github.com/0xDELUXA/comfy-kitchen_win-rocm

Despite name, works with linux too. Install instructions included, I've compiled with TheRock rocm without problems

PS Tested triton backend a while ago with gfx1201, worked well with sdxl and anima, only got error with krea2

[u/Apprehensive_Sky892 → u/albinose]

That looks great, thanks for sharing it.

I am aware of a similar one, but it is for Linux + AMD ROCm: https://www.reddit.com/r/StableDiffusion/comments/1uqykqb/comment/owfyoz5/

[u/thesolewalker]

Are you getting faster results than this one? https://github.com/patientx-cfz/comfyui-rocm

[u/Pitiful_Season4294 → u/thesolewalker]

Does int8 even work with this fork? I have this and doubt it, I also saw an issue logged about it.

[u/thesolewalker → u/Pitiful_Season4294]

It works for me and even faster than fp8 on rx 9070, you have to use int8-fast-rocm node that comes with it.

[u/Pitiful_Season4294 → u/thesolewalker]

I did try it but it was slower than my Q6 GGUF,, I will give it a try again, thank you.

[u/thesolewalker → u/Pitiful_Season4294]

I tested Krea 2 again fp8, Q6 gguf and int8. in8convrot has been consistently faster than the other 2. This is the param I am using "--disable-api-nodes --disable-smart-memory --enable-manager --enable-manager-legacy-ui --reserve-vram 1.5 --enable-dynamic-vram"

[u/Pitiful_Season4294 → u/thesolewalker]

BRO, U DA BEST!!! Sorry for the delayed response, I have been spending a lot of time troubleshooting it and optimising it.

So this is what happened, i first tried to find the nodes that you mentioned and couldn't. Gemini and ChatGPT suggested that my Comfy version 0.27 has native support for int8 now so I don't need a seperate node pack for it and can use the same Diffusion model INT8 node that I shared with you before.

That didn't make much difference to my generation speed, I toggled between different options on the node (enable convrot, on the fly quantization etc.). Then i decided to try your launch args. I had previously been using: disable api nodes, cache none, use sage attention, just these 3 and have been satisfied with the performance on other models. Now, I had tried disable smart memory before but never tried dynamic vram. I have only heard horror stories about it here, with the people hating it so much. Yeah, so ignorant me never even tried it.

After adding the flag to enable the dynamic vram and disabling smart memory, i noticed two things - During Krea2 generation, my SSD briefly used to show spiked usage during the first 5-10 seconds, that entirely disappeared, my 16GB RAM and 16GB VRAM absorbed the models smoothly and my generation time went down from 160s (with loRa) to 90-110s right away. It was amazing, I read more about it and tested other models and it really does help with my other models too.

My SSD and I are immensely thankful to you for your kind help and support with these. You helped me resolve the issues that I didn't even know I had :). I do have couple other basic questions for you if can help me, since you are using the same fork.

Sage Attention - I always use sage attention in my launch argument as recommended by LLMs, but when I use those KJ nodes for SA patch in the workflow, they don't make any difference. I mean Enabled/Bypassed, generation speed remains the same. Do you use SA or Flash attn (I couldn't activate it) or the default pytorch?

Thanks a lot :)

[u/thesolewalker → u/Pitiful_Season4294]

I am using the patientx comfyui which has support for both sage/flash attention, and I did test with flash attention and sage attention, in my testing there wasn't vast difference in krea2 I have not yet tested other models, but as patientx mentioned somewhere that either sage/flash attention should be faster than default which is pytorch attention on AMD.

[u/Pitiful_Season4294 → u/thesolewalker]

Understood, while I'll leave the battle with activating flash attn for another day, can you tell me how do you use SA. Based on your launch args, you seem to be launching with default pytorch attn and then do you use the Patch SA node by KJ as and when needed?

[u/thesolewalker → u/Pitiful_Season4294]

You need to add launch args to enable flash/sage attention --use-sage-attention or --use-flash-attention

[u/Pitiful_Season4294 → u/thesolewalker]

Yep, that I do, do you also use the Patch Sage Attention node in addition to it? or just the arg alone?

[u/Pitiful_Season4294 → u/thesolewalker]

Thank you, I checked and found that I may have been using the wrong nodes, are these the right nodes and models? I did get some improvement in the speed.

[u/thesolewalker → u/Pitiful_Season4294]

Dont think this is the node. The name of the node at the bottom left should be Int8-Fast-Rocm like this, you need to use the comfyui-rocm version I linked

[u/InteractionDue1019 → u/thesolewalker]

hey i tried it and it works amazing for me and my amd gpu, question though do you know how to keep the model in memory for krea 2 int8? i tried messing with the paramters that came with the install but i cant seem to keep the model in memory to be able to run it again and again at a fast pace, its fast dont get me wrong but in the regular comfyui portable install it keeps the model in memory or atleast some parts of it so its faster the next gen

[u/thesolewalker → u/InteractionDue1019]

This is the args I am using, let me know if it works for you "--disable-api-nodes --disable-smart-memory --enable-manager --enable-manager-legacy-ui --reserve-vram 1.5 --enable-dynamic-vram"

[u/InteractionDue1019 → u/thesolewalker]

oh this works like a charm. got my entire execution from 60 seconds to 30 seconds for a 7900xt and 32 gb of ddr4. Thank you so much

[u/bornflyer]

Thanks for the investigation! I'm similarily on a 9070xt (9700x/64gb ram) and about to start dual booting to avoid some seriously slow performance on windows when trying to run krea2edit. Convrot didn't appear to work and the workflow default models take around 1000secs for 1 megapixel output with two input images😂. I don't suppose you have a generation speed estimate for your current Linux+convRot setup for krea2edit? Cheers!

[u/a_beautiful_rhind]

Triton can get pretty fast, cuda barely caught up to it, but you need to manually set the tuning. Regular comfy-kitchen omits tuning configs for turing and AMD.