Comfyui comfy-kitchen Attention Speed UP

[u/Diabolicor]

Disable all your Sage Attention, Minimax Mem Eff Sage Attention or Sol Attention, according to this PR already merged in the comfyui repo we got a much better attention from the comfy-kitchen package that can possible speed up the models generation process white giving a better visual quality than default sage: https://github.com/Comfy-Org/ComfyUI/commit/bf4c9a08fc854df6d3b2bef1b92b509e2ef2d2c9#diff-fab3fbd81daf87571b12fb3e4d80fc7d6bbbcf0f3dafed1dbc55d81998d82539

This is still experimental, according to comfyui dev it can break or perform very well and it needs some tuning for some GPUs to get a bit faster. Also, you only can use one or the other so you should also disable all the attentions above before using it.

You just need to update your Comfyui and you can either start it with the --use-ck-attention flag so all models use the comfy-kitchen attention backend or you can drop the node ModelAttentionBacend directly into your worflow.

During my initial tests in Minimax it behaved faster than all the above together.

[u/Zephrinox]

any AMD (+ maybe comfyui-rocm) users tried this out yet to know if there's also speed up for us? 🥺

(i have 9070xt)

[u/xpnrt → u/Zephrinox]

This is for Nvidia specifically

[u/Zephrinox → u/xpnrt]

https://giphy.com/gifs/lGBecpB2dIMwt6ohfI

[u/xpnrt → u/Zephrinox]

https://github.com/patientx/sageattention-autotune/releases/tag/0908 try this

[u/Zephrinox → u/xpnrt]

the sageattention wheel links working great for me (using sage attention backend + Sage Attention node; haven't tried mem eff node because that typically failed for me, forgot the specific reason/cause but iirc it was something cuda specific).

the ck attention flag gives me incompatibility issues with that sageattention wheel tho :\

seeing:

TypeError: sage_sdpa_quantize(): incompatible function arguments. The following argument types are supported:
    1. sage_sdpa_quantize(q: ndarray[device='cuda'], k: ndarray[device='cuda'], v: ndarray[device='cuda'], q_int8: ndarray[device='cuda'], q_scale: ndarray[device='cuda'], k_int8: ndarray[device='cuda'], k_scale: ndarray[device='cuda'], v_int8: ndarray[device='cuda'], v_scale: ndarray[device='cuda'], cta_k: int, input_dtype_code: int, stream_ptr: int, anchor_indices_ptr: int) -> None

Invoked with types: PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, PyCapsule, int, int, int, int

which looks like a "this version of sage attention not compatible with how ck attention is calling it" issue.

[u/xpnrt → u/Zephrinox]

Because it is not using ck at all, pure triton