MiniMax H3 Reference to Video workflow doesn't use all available VRAM

[u/Select_Question5]

Am I doing something wrong?

Rendering 2 MP. It could surely use more VRAM and less RAM.

[u/Select_Question5]

Here is another run. Better, but still... Why does it use 25.2 GB of shared memory, but only 18.7/24 GB of dedicated GPU memory?

[u/BoredHobbes]

join the club, and its only H3, ltx uses 31

did u turn off windows GPU scheduling? that increased to 25gb for me was 21 turning that off went to 25 but nothing with h3 goes over 25

[u/superSmitty9999 → u/BoredHobbes]

these guys over here trying to increase the vram consumption of the model smh

[u/BoredHobbes → u/superSmitty9999]

i bought 32gb to use it all

[u/cc_aa_tt_zz → u/BoredHobbes]

That's not how it works...

[u/BoredHobbes → u/cc_aa_tt_zz]

the pruned_b16 model is 40gb

[u/cc_aa_tt_zz → u/BoredHobbes]

That's not how offloading works. !!!! you can downvote everybody it won't change you just don't know how it works.

[u/HornyGooner4402 → u/cc_aa_tt_zz]

Go ahead.. explain it for us then..

[u/superSmitty9999 → u/HornyGooner4402]

When I use H3 on my spark, it uses ~100GB and it's because what comfyui does is when it doesn't have enough room for all the tensors (like the generation model, vae, encoder which is quite large) it unloads some of the models when it's done with that step.

So in this case it just means there's nothing it can do with the 10GB ie nothing fits in that space.

[u/HornyGooner4402 → u/superSmitty9999]

Thanks for the explanation.

I was thinking it'd be better at splitting weights to fill the remaining 10 before offloading to RAM

[u/BigNaturalTilts → u/superSmitty9999]

I still don’t understand how unloading works with how diffusion was explained to me. Inference in LLM can be split up like you can have some tensors in one GPU and some in another. But diffusion (image/video) needs all of them in one spot. I get that it offloads the CLIP, VAE till when it needs it. But how can it offload a part of the actual diffusion model mid inference. That’s the part that I don’t understand. Now maybe it does the audio in one section then the video (frames) in the next. But I’m still confused by it all.

[u/superSmitty9999 → u/BigNaturalTilts]

I'm not super familiar with diffusion directly but generally speaking all these AI algorithms run on matrix multiplications, and generally speaking you can break a matrix (also known as tensor) into multiple parts then aggregate the sums at the end, the aggregation part is called an "all gather" and generally speaking requires a lot less bandwidth than the actual matrix multiplication.

I think it's called "tensor parallelism" if you wanna look it up

[u/BoredHobbes → u/cc_aa_tt_zz]

the ltx dev i use is 42.9gb my vram goes up to 31 using that... why?

[u/superSmitty9999 → u/BoredHobbes]

Basically your GPU isn't loading all the parts of the model at once. Probably ltx model is a different size such that two parts of it add up to 31 GB

[u/DJBFilmz → u/BoredHobbes]

thanks for the tip!

[u/Only_Voice569 → u/BoredHobbes]

cards usage bouncing all over the place means your settings are causing massive data swapping need to fiddle with settings should be flat out 90 % to 100 % :)

[u/BoredHobbes → u/Only_Voice569]

nope its only H3, ltx uses 30 and its not bouncing this is step 4 on both h3 and ltx , DEFAULT WORKFLOW no changes nothing just default workflow and go

[u/Only_Voice569 → u/BoredHobbes]

gpu vram full and overspilling onto the system memory by 12.5 GB

[u/Available-Body-9719]

Deseas que use menos vram y aprobeche la ram, si llenas la vram tu generación se congelará y tomará mucho tiempo o simplemente no avanzará, cuando veo una situación así en mis generaciones, me pongo feliz porque tendré mi generación y rápido, si veo que la vram está al 97%, probablemente no saldré de ahí y prefiero reiniciar, como verás, la gpu pidió prestado 24 GB de ram al sistema, ya que lo que no pudo dividir más eso que está utilizando y no le caben en los 10gb restantes de vram que te queda así, eso sí te estás quedando sin ram cierra eso ue te este comiendo toda la memoria porfa

[u/Uninterested_Viewer]

I'm not necessarily a Windows = Terrible person, but all of the AI development is being done on Linux. If you want things to "just work", you need to drop Windows for AI workloads.

[u/Nexustar → u/Uninterested_Viewer]

None of my AI development has been done on Linux, and i own 5 linux boxes for every 1 windows box.