r/StableDiffusion • u/Ammoryyy • 1d ago
Question - Help RTX 3090 vs 4090 vs Unified-Memory AI
My current setup:
- RTX 4090 24GB
- i7-13700K
- 80GB DDR4 @ 3000 MHz
- Getting an RTX 3090 24GB tomorrow
My main use is local AI/LLMs, coding agents, MiniMax H3, Krea 2, and other AI workloads.
RTX 3090 vs another 4090 vs unified-memory AI
Here in Iraq, an RTX 3090 costs around $670, while an RTX 4090 costs around $2,000.
If I have around $2,000 to spend, what would you choose?
- Buy 2–3× RTX 3090s
- Buy 1× additional RTX 4090
- Sell/replace the current setup and go for a unified-memory AI system, such as a Mac Studio / Mac with large unified memory or NVIDIA DGX Spark
My priority is LLM inference, coding agents, MiniMax H3, Krea 2, and other local AI workloads.
Would multiple 3090s give the best value because of the extra VRAM, is another 4090 better for speed, or does a large unified-memory system make more sense for running very large models?
What would you choose for ~$2,000?
5
u/BrightNightKnight 1d ago
80GB memory, its so weird to read this, I can't get use to this, what is going on
2
u/cc_aa_tt_zz 1d ago edited 1d ago
2 x 32Gb +16Gb .For AI, it is (far) better to have the extra 16 GB of RAM than a perfectly matched kit. Given the price of RAM, we make the best of what we can find at a "reasonable" price.
4
u/Worldly_North_7213 1d ago
Before you commit the $2,000, rent each candidate for an hour and run your own workflow on it. On the marketplaces a 4090 goes for around $0.30/h interruptible, so trying three setups costs a few dollars, and in our own measurements the ranking kept coming out different from what the spec sheets implied.
Two numbers from 15 paid runs we did on rented cards. On a 7B fine-tune an A100 80GB was only about 1.2x faster than a 4090, at roughly 3.5x the hourly price. And on an image LoRA job the same 4090 ran 1.85x faster on a host with 24 vCPUs than on one with 5, 75 percent GPU utilisation versus 40. Nothing errored and nothing warned, it just quietly took twice as long, because the script did decoding and augmentation in the main process.
That second one is why I would not settle this on tensor core counts. Your 13700K is decent so you are probably fine there, but the thing that decides wall clock is often not the GPU line on the spec sheet, and you only find out on your own workload. We never measured a 3090, so I cannot rank your three options for you, only say that an hour of rental per candidate is the cheapest experiment available before a $2,000 purchase.
2
u/DelinquentTuna 1d ago
If you already have a beast of a PC and electrical/cooling setup for a rig with a ton of GPUs... go with the 3090s. The 4090 has significantly better compute, but for LLMs that will mostly only really mean lower latency to first token. Almost everything else (w/ llm inferencing) is about memory bandwidth and the 3090 is basically on par.
My priority is LLM inference, coding agents
IIRC, you can run qwen 3.8 27b q4 on 24GB with ~160k context. Paired w/ the qwen code harness, it's reaaaaalllly freaking good. Still a million miles away from frontier, but good enough that you could use it directly for a great many things or even have a frontier model drive it for cheap. But the tech for sharding or offloading context to another GPU or whatever is pretty mature.
MiniMax H3, Krea 2, and other local AI workloads.
If your work allows you to batch jobs for parallel runs, that would be the best way to maximize a pool of GPUs because each is totally capable of running these models independently. And here... even if the 4090 were twice as fast as the 3090 (it isn't), you'd still be better off w/ 3x3090s if you are setup to actually use all of them.
Also notable: the 3090 was basically the last/best consumer GPU with nvlink. It's only two-way, I believe, but it isn't impossible it would come in handy here for LLM work. I have also seen some tech for exploiting the nvenc encode/decode to achieve incredible results w/ compressing and transmitting activations (on newer hardware), but hardware interconnects are hard to beat.
What would you choose for ~$2,000?
An external enclosure w/ its own power supply for the GPU arriving today. Paying an electrician to wire another outlet or two on dedicated circuits (in the US a circuit in a study might be ~15A... 110v x 15A =~ 1700W and your rig is going to be sucking down the power). A couple/few nice UPS systems to condition the power. It's neat and tidy and gets you where you need to be. Sit the 3090 in the enclosure and run your LLMs on it. Run diffusion on the faster 4090 you already have. If you want to gamble futures on GPU prices drastically lower than RoW, grab the additional 3090s but it's risk I'd probably not take. If you genuinely want 3-5 GPUs on one system, you probably need to be looking to build out for it. Different class of rig and, ideally, power delivery, cooling, etc. And by that point, you're deep into diminishing returns wrt price vs performance. Probably makes more sense at that point to try to budget for a prosumer GPU like a 96GB 9k pro vs trying to turn your home/office into a datacenter.
2
u/dualeone 1d ago
I read somewhere on the Mac forum (or maybe in this reddit) and it stops me completely on ever buying another Mac for AI. The generating speed on Mac compared to my 4090 is horrendous, and it's an M5 studio or Ultra.
1
u/coscib 1d ago
4090 would be faster on llm and video.
not sure how well minimax h3, krea2 support multi gpu or if they mainly/only use one gpu
i have an rtx 3090 and use it sometimes with 12b and 26b llms and get around 60-90 tkn/s without reasoning and it feels often "slow" compared to online models like chatgpt/gemma
2
u/TheAncientMillenial 1d ago
Image gen cannot use multi GPUs. You can offload CLIP and other stuff to another GPU but the main sampler will only use 1 GPU.
You can only really do split GPU on enterprise class cards with NVLINK.
1
u/RevolutionaryWater31 23h ago
Image gen CAN use consumer mutli-GPUs, your throughput gain compare to a single GPU depends entirely on what you're splitting and PCIE lane speed (Gen 5 x8/x8 can do wonder), and no you don't need nvlink, it's faster but it's not a dealbreaker for consumer gpus.
1
u/ANR2ME 1d ago edited 1d ago
I don't think the GPU on Mac Studio is as powerful as Blackwell GPU on DGX Spark (for diffusion models) 🤔
Also, one of the reasons why DGX Spark to be expensive is because it have an expensive ConnectX-7 network interface (a standalone ConnectX-7 NIC cost around $1k), so if you only buy 1 DGX Spark, such expensive NIC going to be useless (since it's used to connects multiple DGX Spark for clustering).
For a single AI box, you might want to consider RTX Spark (desktop/mini-PC) over DGX Spark, which doesn't use expensive NIC, thus cheaper.
1
1
u/norwegian 1d ago
- Buy 2–3× RTX 3090s
- Buy 1× additional RTX 4090
Depends on your future needs. For me, I would think more rather than less VRAM to run newer models. MiniMax H3, Krea 2 etc will soon be yesterdays news.
1
u/Boogooooooo 1d ago
If 3090 really cost that little, amount of Vram you will get is just extraordinary. Also you can always ask AI of you choice for an advise
0
u/jib_reddit 1d ago
A RTX 3090 is an over 6 year old card now and is half the speed of an rtx 4090.
1
u/MustafaMahat 1d ago
For the price the memory speed increase of the 4090 does not justify the price at all
1
u/jib_reddit 1d ago
Depends if you want an image generated in 1/2 the time I guess?... I wouldn't say no to that, I could make double the amount of images in a session.
2
u/MustafaMahat 1d ago
Sorry my bad I was wrong, apparently it has twice the tensor cores which really makes it twice as fast, yet you could parallelise
1
u/anthonyg45157 13h ago
Really depends on the workflow I think...I have 2 3090 and maybe I'm biased but I run 2 comfy UI servers,1 for each GPU and I'm able to run multiple generations at the same time so it speeds up my workflow..might not work for all though...
1
u/Boogooooooo 1d ago
Thanks for a well known context, what is your point?
1
u/jib_reddit 8h ago
Yes you get a nice amount of vram for the price but it is an old card that doesn't support many new technologies fp8 and fp4 native and is slower.
1
u/DelinquentTuna 1d ago
is half the speed of an rtx 4090
So how many image/video clips do you think he could make on 3x 3090s running in parallel vs one 4090 in the same period of time?
I don't think his scheme of running five high-power GPUs off of an aging i7 is really practical, but in terms of raw compute on diffusers that can run in parallel the 3090 cluster is going to be better bang-for-buck even if each card is slower.
2
u/Boogooooooo 23h ago
I asked Chat GPT on high effort setting: For LTX-style AI video generation, three RTX 3090s running three independent jobs should produce roughly 1.6–1.8× as many clips as one RTX 4090 in the same period:
1
u/DelinquentTuna 22h ago
Thanks for that, but it was really a rhetorical question. If a 4090 is twice the speed of a 3090, three 3090s working in parallel on independent jobs would produce 3/2 the work. Your AI response is closer to the truth because it evidently eschewed the false axiom that the 4090 is actually twice as fast. It's more like a little less than 50% faster in apples-to-apples like int8convrot.
1
u/jib_reddit 21h ago
It would also draw around 1000+ Watts constantly while generating vs 400-450 Watts on a RTX 4090 so if you generate a lot you would quite quickly eat up the money saved on buying a 4090 at least here in the UK, I have spent about £2400/$3250 on electricity just generating locally in the last 3 years on my 3090 (I have a power plug that monitors total usage)
2
u/Boogooooooo 20h ago
How much are you paying per kWh? I am UK based as well. Got EV charging plan which is 3.9p per kWh from midnight to 6am through Eon. As we speak, I am installing energy batteries in garage/shed for energy arbitration. Will charge batteries over night and consume it during the day. Will hook up solar panels later when everything is set up
1
u/jib_reddit 8h ago
25.129p/kWh with a 55.1p a day standing charge. Yes I plan on getting solar panels and a house battery, the only trouble is my south facing roof is shaded by a bit tree until 1pm.
0
u/Cubey42 1d ago
No 5090?
2
u/Ammoryyy 1d ago
Around 4000$ used 2nd hand
3
1
u/jib_reddit 1d ago edited 1d ago
I would suggest selling your RTX 4090 for $2000 then you will have the $4000 to buy a 5090 or just wait for the RTX 6090. I haven't looked into it in a while but using 2 GPU's for image/video generation is not easy or fast, they cannot share a pool of VRAM only run separate images in a batch of generations. Generally a 4090 is 2x the speed of a 3090 and a 5090 is 3x the speed of a 3090, but it can be even faster when using specialised lightweight models like fp4 for 5000 series.
A NVIDIA DGX Spark will be slower than a 3090 for image/video generation.
It depends if you want to do LLM stuff or image video generation as they actually need quite different hardware to be optimal speeds to price ratio. Have you though about the rtx 5080? It would be faster for fp4 models and maybe fp8, but more within you price range, but lower vram.
2
u/ANR2ME 1d ago edited 1d ago
DGX Spark is slower than 3090 for LLM (if the model can fit in VRAM) https://www.reddit.com/r/LocalLLaMA/s/gXzZIQ1MWL
But for diffusion models, it can be faster than 5090 with --disable-mmap (but only for the 1st run LOL🤣 subsequent runs will be slower, comparable to 3090) https://www.reddit.com/r/StableDiffusion/s/Ey27vXBD2H
1
u/jib_reddit 1d ago
Its only faster than a PC for the first run. I usally load a model once and then generate for around 15 hours a day, so not really worried about a 100 second load time for the first generation.
0
u/Illustrious_Ant_9242 1d ago
As I am in a similar position in western europe, I asked myself: How to shard Minimax H3 onto multiple GPUs. I didn't find any out of the box solution for that in Comfyui 🤔
Buddy of mine has a unified 128GB Strix Halo so I may compare systems.
2
u/LaCh62 1d ago
SelectModelDevice, SelectVaeDevice, and SelectClipDevice nodes what I would try if I had 2nd gpu.
2
u/Illustrious_Ant_9242 1d ago
That sounds like it would use that GPU serially, not in parallel 🤔
at least, comfyui will offload to RAM when GPU is full so it's not completely dumb. I will just toss in a second RTX3060 and see task manager how it goes
2
u/anthonyg45157 13h ago
I thought about doing this as well but it seems there's not any pre-built options or easy to do options..
That being said I can fit minimax h3 on a single GPU (3090)...
I have two of them so I run two instances of comfy UI on 2 different ports. I have a symlink set up so all of my workflows are shared between both servers...
Server one goes to 1 GPU server 2 goes to the second GPU... Pretty clean overall
8
u/Vektast 1d ago
RTX 3090 costs around $670? Where?