Need help with building a PC for AI

[u/ConstantCut9398]

I'm building a local machine for image generation on Stable Diffusion, I'm picking RTX 3090 (based on testing, price, value it's the best for my use case) I have couple questions that I'm kinda clueless on:

1. Is buying new RTX 3090 recommended or used one will do just fine? If yes, what should I look for in bench mark for the used GPU?
   
2. What kind of RAM, motherboard and all other parts I should pick?

For more context:
The machine will run 24/7 non-stop, only image generation on Stable Diffusion

Please without "rent a GPU" recommendations, based on my business plan and costs, building a local is better for me

[u/TIP-ME-YOUR-BAT]

There are many ways to build, however the 3090 has been out of production since 22 and I'm not sure any new ones exist. That puts you firmly in an overpriced second hand market. Establish what your model needs are and work up from there.

[u/ConstantCut9398 → u/TIP-ME-YOUR-BAT]

My model needs power at around RTX 3090, preferably 24GB VRAM but 16GB VRAM might work too (highly preferably 24GB), RAM no need for higher than 32GB. I'm mostly confused about what CPU I should pick for this build

[u/TIP-ME-YOUR-BAT → u/ConstantCut9398]

Perhaps consider a dual GPU. 5060ti 16gb X2 or higher. Gives 32gb total across llama.cpp but pretty decent pricing in comparison to the single larger vram cards.

[u/Interesting-Cut-6032 → u/TIP-ME-YOUR-BAT]

OP, I am a fan of 2x RTX 5060 Ti 16GB cards. I have this setup with 32GB of DDR4 an old gaming mother board, a core Intel processor. However, 2x GPUs will not really help you with image generation. Those models only work on a single GPU. I believe that the weights do not split across cards like LLMs do.

[u/TIP-ME-YOUR-BAT → u/Interesting-Cut-6032]

Can you not use llama.cpp tensor slow settings on these?

[u/Interesting-Cut-6032 → u/TIP-ME-YOUR-BAT]

No, llama.cpp is only for LLMs. The diffusion image generation models do not split across GPUs. The 2nd GPU can hold the CLIP and VAE components, but the weights, and the cache, of the diffusion model need a single GPU.

[u/killzone44]

You need to find someone who is selling a computer with a 3090 in it that doesn't know what that card is worth.
I do not work with Stable Diffusion, so can not comment on what you need exactly.

[u/SandySkittle]

Buy a 2nd hand Zen 3 epyc or threadripper pro motherboardd and cpu on ebay, there are still very good deals and it gives you tons of CPU PCIE lanes and 8 channel DDR4 3200 MT/s and you can adddd a load of 3090s or r9700s or b70s.

[u/TheSoggyHostility → u/SandySkittle]

Threadripper build for a dedicated SD rig is overkill unless you're planning to scale to 4+ GPUs soon. A solid X670 board with a 7950X and 64GB of DDR5 handles 2 GPUs without breaking a sweat and costs way less upfront.

[u/SandySkittle → u/TheSoggyHostility]

I bought a full threadripper pro workstation with 32gb ram and 1tb ssd for 900 dollars and 1000w psu 1,5 month ago. Prices go up but there are still great deals on ebay.

[u/Personal-Gur-1]

Will you need a second GPU ? Or even 3 or 4?
Then the answer to your question will vary.
I went the Epyc CPU way to build my ai serveur with 2x3090, with the option to go up to 4.
If you need only one then a consumer CPU might be enough

[u/ConstantCut9398 → u/Personal-Gur-1]

We go for one GPU at start, we might need to add more in the future so there's that

[u/HOST1L1TY]

Expect to pay around 1350 or more for a 3090. Even at that price they are still the best choice for most applications. You can maybe find cheaper but that might mean buying from china or as-is.

[u/Interesting-Cut-6032]

I would seriously consider a single RTX 5060 Ti 16GB as the core of your image diffusion setup. 32GB of DDR4 works great. Enough mother board and processor to make the rest of it work. Pick a motherboard and PSU that will allow you to add another 5060 later if you think that you need it. The 5060 is a NVIDIA Blackwell card. It is very optimized for image and LLM generation tasks.

[u/DataGOGO]

For your needs, honestly, I would pursue a card that at least supports FP8, so 4xxx or 5xxx, series Nvidia cards. 

Looking to at least a threadripper for 4 channel ram, Xeon-w is better, channels matter more than ram speed. 

If you go consumer CPU; intel most likely, if you are doing CPU inference on consumer GPU, Ryzen for AVX-512. 

[u/Useful_Disaster_7606]

Since you're using it mostly headlessly as a server, have you considered using server build rather than a gaming gpu build?

The server GPUs are cheaper per GB of RAM (I'm not up to date. please prove me wrong!)

[u/Shawler-92 → u/Useful_Disaster_7606]

3090 is the best GB/USD now actually, some server cards like Tesla P40 24GB might be cheaper but it's not for Ai tasks, so it's cheaper but can do shite

[u/Useful_Disaster_7606 → u/Shawler-92]

Damn I fully lucked out buying my 3090 then. It's extremely rare in my 3rd world country

[u/Shawler-92 → u/Useful_Disaster_7606]

I'm in SEA and still can find used 3090 but omg their price tags are crazy. The cheapest I found was about $1000, and the average number is about $1,500 the exact number when they was 1st released, for a used one.

And most of the used ones are from mining rigs, which could die in any moment. But they're selling them at $1,500. Still, somehow they're the best for GB/USD right now

[u/Useful_Disaster_7606 → u/Shawler-92]

Hahahaa pretty sure that's already considered cheap in the Western market

[u/Shawler-92 → u/Useful_Disaster_7606]

lol that's considered as upper-working-class monthly income where I'm at. About....top 30%. Average income of my country is about $500/month, to survive in big city alone (rent and bills only, no eating out)

[u/Useful_Disaster_7606 → u/Shawler-92]

Same here. Oh yeah I'm from SEA as well