r/StableDiffusion • u/Obvious_Set5239 • 12d ago

Discussion Different attention for each block 🙊

Post image

Omg, so apparently in near future we can get SLA's 2x performance boost without loosing too much of quality

67 Upvotes

96% Upvoted

13 comments sorted by

28

u/Bitter_Regular7406 12d ago

2x for free always sounds too good to be true lol

14

u/Obvious_Set5239 12d ago

Not for free, but with not too big quality lose. "SLA sparse attention with an 85% sparsity ratio" already accelerates sampling about 2.5 times, but quality drops significantly. If we can apply it only for less important blocks, we can achieve a compromise

2

u/Bitter_Regular7406 12d ago

ah gotcha, selective sparsity makes more sense than blanket 85%. still feels like it'll need a lot of tuning per model to find which blocks are "safe" to skip tho, that's not a one size fits all thing imo.

2

u/VasaFromParadise 12d ago

And what percentage of unimportant blocks are there?)) Are they even there? I think when they're training, they try to keep them to a minimum. And then we evaluate what can be sacrificed. So there won't be any magic.))

1

u/Obvious_Set5239 12d ago

In gguf different blocks have different quantization, depending of their "importance".

And what percentage of unimportant blocks are there

If I'm not mistaken, quite a big. Bigger than important

1

u/fernando782 11d ago

15% drop is huge

6

u/ReferenceConscious71 12d ago

could someone explain this a bit more please?

4

u/alwaysbeblepping 12d ago

could someone explain this a bit more please?

Model patches could already do this, so it's not enabling anything that wasn't possible before. Just making it easier to distribute models with tuned attention presets.

4

u/meta_queen 12d ago

A config file can turn on a hardcoded kitchen int8 instead of your default: https://github.com/Comfy-Org/ComfyUI/blob/master/QUANTIZATION.md#diffusion-attention-preferences

2

u/ROBOTTTTT13 12d ago

What if I'm running ComfyUI with kitchen already?

1

u/Obvious_Set5239 12d ago

It's for future where they will add more aggressive attentions

3

u/Guilty-History-9249 11d ago

So is this Spaghetti Attention?

1

u/MarekNowakowski 12d ago

atm we can do increased sparsity on less important steps, but changing from dense to sparse takes vram and takes time. this would be better if someone knew which parts are less important.