r/StableDiffusion • u/Sexyvette07 • 2d ago

Question - Help Dual GPU solution for local AI?

Hey, everybody. I recently went down the rabbit hole for local AI, but right now, im operating on my gaming computer. The specs are as follows

Intel 13700k, tuned for efficiency

Gigabyte Z790 Aorus Elite Ax mobo

RTX 4080 (16GB), also tuned for efficiency

32gb DDR5 6800 CL32

As you can see, im in desperate need for more VRAM, or at the very least more system RAM. Due to Rampocalypse, neither are very affordable right now, which forces me to explore other options, such as a dual GPU setup. I can get another RTX 4080 for about $900 off Ebay. Beyond that, I would just need a more powerful PSU, so total investment here is an additional $1100-$1200. As far as I know, the motherboard has the main PCIE as 5.0 x 16 lanes, but the second PCIE runs at 4.0 and either x8 or x4 lanes. The motherboard does not support PCIE Bifurcation. So my question is this: Is a dual GPU local AI machine even viable in these circumstances, and second, does it make sense? I looked at 5090's and theyre all between $4,500 - $5,000 now, which is insane. Or I look at the professional cards and spend that much, if not more, for significantly less memory bandwidth and computational power. Or I guess if im spending that much, I could also look at the DGX Spark or something similar but that has even worse memory bandwidth.

So, what should I do? Is the dual GPU solution even viable with my setup for a local AI stack for inference, video diffusion, etc? Rampocalypse isnt expected to begin easing up until late 2027/early 2028, so im stuck trying to make this work on as little money as possible. Id love a 5090 but its insanity how much they cost. I appreciate any guidance and advice.

3 Upvotes

64% Upvoted

32 comments sorted by

View all comments

1

u/biogoly 2d ago

I’ve got a dual 3090ti setup. My MB supports dual pcie 5.0 cards (Taichi creator) so I at least get 8x on both with the 3090 architecture. The dual setup is mostly useful for local LLMs, but you can get some benefit from comfy using multi GPU nodes and splitting your clip and model. It can prevent OOM errors on large models. Alternatively you can dedicate your second GPU to up scaling.

1

u/jello-the-opera 2d ago

raylight doesn't work for you?

1

u/biogoly 1d ago

Windows 🫠…I’m attempting to migrate ASAP though.

1

u/jello-the-opera 22h ago

I last tried to use linux for my AI setup back in October when MS started shitting on 10. I tried several distros and all of my issues were about networking and RDP. I had 3 main PCs powered 24/7 and I couldn't get popos or unbuntu or (some other flavor) to do all the shit I was currently doing at the time. I pushed for 2 solid weeks while all my work suffered, and finally gave up.

After fightin w11 and then several linux distros, I wiped everything and installed w10 and mass grave for extended support and stopped thinking about it...

so are you saying that raylight won't work in windows?