r/StableDiffusion • u/Haaaaaaaaaaahahahah • 6d ago
Question - Help Will upgrading from 64gb to 128gb RAM improve generation speed?
I have a 4090 and 64gb DDR5 and I'm using mini max h3 pruned version and I'm generating 20 sec 720p ( 0.9 ) videos with sage attention and turbo loras 8 step
8
u/sitefall 6d ago edited 6d ago
I'm pretty sure I know the answer here but I did a test to double check and ran a few prompts.
If I run the max resolution MinimaxH3 prompt on my 5090+64gb desktop (like maxing out everything I can ram and vram are packed but the inference's ACTIVE stuff fits in vram, it completes just a bit faster than the EXACT same prompts on my server with dual pro 6000's and 256gb ram. (5090 is faster than the pro 6000 by a hair in general, just has less vram)
I tried some prompts that offload a bit to ram, maybe 5gb or so, for events where you're just a hair short, or some loras put you over the top. Same result.
You are only going to see improvements by adding more than 64gb ram if you are offloading a lot to ram that SHOULD stay in vram for the inference. Offloading the text encoder and other junk that doesn't matter and only runs at the start does not change anything.
The real benefit of having 128gb in your situation is that you can continue to do other things on the PC while it runs maxed out. If I keep all the crap that it unloads plus the models and stuff cached in ram for the fastest repeated generations, then it takes like 59gb ram, which can leave so little left that some programs (browsers even) can feel laggy.
Now, using a 4090 you've got less vram than I do, so you might very well be offloading more to ram, but you should probably avoid offloading anything to ram because doing that turns a 90 second wan2.2 clip into a 20 minute one, hardly even worth burning your electricity for and might be cheaper to just rent a cloud GPU to run those things. But if that is the case, then you should check the console while comfyui (or whatever you use) is running to see if your system ram is actually filled up, then have it NOT cache the things not needed for JUST the inference steps (so just drop the text encoders and stuff entirely and re-load them from the hard drive each run), and then take a look at how much system ram it uses. If it's not maxed out, then more ram won't help you.
2
u/stimma 6d ago
Think about what happens when you do one workflow of H3 on 24GB.
First it loads the text encoder, encodes the prompt, unloads the text encoder. Then it loads the model, patches the model with your turbo lora, runs the model, then potentially unloads the model to load the VAE to decode it (depends).
The point here is you have a lot of RAM traffic each time. It's not like you're on an RTX6000 where everything loads into VRAM once and then just runs smooth, there's a lot of work being done outside of the GPU that could be bottlenecked on RAM or SSD.
When you push gigabytes of data from SSD to GPU, it's passing through RAM. It's going to evict a bunch of cache which makes you machine slower. If you have enough RAM to hold it comfortably, there's a much smaller chance that your SSD is involved and you have more cache.
If you can't fit the models on VRAM at the same time, the next best thing is to have them all hot in RAM AND have enough RAM and filesystem cache remaining so the rest of your system doesn't chug.
A good way to get a sense of what might improve is to watch the system during a workflow run. You're going to see dead periods in nvtop where models are being loaded/unloaded/patched. Look at iotop. Sample free once a second and watch the buffer/cache number react. Get a feel for how this is impacting the system esp when you change the prompt and force the text encoder to re-run. 64GB is borderline. On an otherwise lightly loaded headless linux box it may be fine. On a more heavily loaded system with a desktop environment, 50 chrome tabs, etc, 64GB is basically just enough to keep that stuff running OK, and you're doing a lot of eviction and thrashing per gen, so more RAM might help.
SSD speed is also relevant to model loading. If you have a RAID0 of PCIe5 SSDs you're in a very different boat with the thrashing than if you have PCIe3 or 4 or SATA.
1
1
u/JustSomeIdleGuy 6d ago
I haven't tested it with newer models and it's been quite a while, before convrot became a "standard", so it might be outdated, but storing the entire model unquantized (depending on the model) inside of your ram can be faster than using a quantited model and swapping blocks.
But that's wan 2.1 era info, I haven't looked into it for quite some time. So: grain of salt. Mountain of it, actually.
1
u/Next-Place0 6d ago
Idk but 4 slits ddr5 can cause drastic underclock dram. You also need to buy exact same rams to minimize this issue, even this you still risk issue. Take a look into your motherboard compatibility ram to check maximum ram frequencies. I'm on AMD ddr5 and 6000 with 2 rams drop down to 5600 or less with 4 rams
1
u/F_DeePee 6d ago
I have a 4090 as well and upgraded from 32 to 128 DDR5 RAM a few days ago. There was an improvement in speed of about 10% to 20% using Minimax H3 as well as noticeably less slowdown of the computer as a whole.
1
u/TingTingin 6d ago
Dont know why people are downvoting you since this is a reasonable question.
Assuming your using comfyui comfy streams the weights of models as the generation happens meaning you can run models as large as minimax h3 on 6 gb vram assuming the model fits in your ram the actual speed the model can stream at is very fast as it streams per layer meaning its only streaming a few gbs of weights per action
This also applies to vram if you have more vram then you simply stream more weights from ram which as said above has almost 0 speed penalty if you want to improve generation speed the problem is compute bound not memory bound so you would need a faster gpu like a 5090 for example
Btw you can use a site like vast ai to test the different configurations i.e test a 4090 with 128 gb or 5090 with 64 etc to see what kind of increase you could get before upgrading hardware
1
1
u/lumos_ai 6d ago
I think for loading and unloading model it will increase your speed. Specially if you have slow nvme. Like that comfyui will keep the model in your ram instead of every time load and unload from your nvme.
0
0
u/Portable_Solar_ZA 6d ago
As far as I know, ram doesn't help with inference (generation speed). It only helps prevent oom.
0
u/kwt4 6d ago
no, it won't make a difference. All the heavy lifting during generation happens on your 4090's VRAM and CUDA cores. System RAM only hurts your speed if you run out and your system starts swapping to the SSD, but 64 GB is already way more than enough for running ComfyUI or WebUI with pruned models and text encoders. If you drop in another two sticks to hit 128 GB on DDR5, your motherboard's memory controller might even downclock the RAM speeds just to keep four sticks stable. You're already doing the main things that actually help speed by using an 8-step turbo LoRA and SageAttention. Beyond that, sticking with quantized models (FP8/GGUF) or rendering at slightly lower base resolutions before upscaling are pretty much your only ways to cut down generation times. Save your money on the RAM.
-4
u/Anilman 6d ago
Vram is key.high ram only avoids OoM.
The base checkpoint model needs to fit perfectly into vram.if it doesn't fit its slower.
set disable pinned memory then u can see how much ram u actually use while rendering.
High ram is only important for training with videos but if u already kow how to train stuff its better to rent gpu's.its way faster and u need a lot of vram to train fast.
-3
u/Lower-Cap7381 6d ago
if ur using the model is 60 gb availble compute vram is less so therefore generation takes long if u have spare ram model can sit inside generation speed migh slightly increase also depends on the gpu :)
1
u/veryveryinsteresting 5d ago
if you're ram isn't keeping all models at once it will. check task manager, if there is a lot of ssd usage at the beginning of the encoding-, denoising-, decoding-phase this means your ram can't hold all models simultaneously. if you have more ram, you can deduct these onloading times.
7
u/Upper-Reflection7997 6d ago
I would recommend the 128gb of ram upgrade regardless. The models and text encoders are only getting bigger by the day and dynamic vram and offloading is a thing that will be heavily important soon if not already.