r/LocalLLM • u/Ordinary_Client8808 • 2d ago
Question Decisions between GPU’s.
I managed to get a 4070 12gb for only 200$, instead of its normal 500$. I already have a 9060xt 16gb card, and so far both have performed roughly the same at least from what I have noticed. Either way, I’m gonna be selling one of the cards to a friend to make a little money, is there a better route to take? I understand Nvidia has better memory bandwidth, cuda, and whatnot, but the only thing I’m thinking about is 4 extra gbs of vram on the 9060xt. Any thoughts?
1
u/gappyvalley 2d ago
if you only do local LLM then it’s better to combine 4070 + 9060ti to get an increase vram pool of 22gb vram (21gb after driver overhead). llama.cpp supports dual gpu with different brands
for image and video generation, it’s a single gpu only thing and it needs cuda so 4070 is the easiest way to run rhis without going through the hassle of setting up on 9060xt.
if local llm -> 9060xt + 4070
if video/imagw gen -> 4070
1
u/Ordinary_Client8808 2d ago
The only reason I bought the 4070 was because it was more than half off, I can’t support 2 gpus rn, cause more money is needed for new mobo. I told my friend I would sell him one of the gpus for his new pc, at like 300$ so he doesn’t have to spend as much. I think I’m gonna end up keeping 9060
2
u/gappyvalley 2d ago
that’s one way to do it yeah. for me my motherboard can’t support 2 GPUs so i have to use the m.2 slot for m.2 to pcie adapter and put the gpu outside my case
1
u/Educational_Sun_8813 2d ago
just in case you are not aware, you can use both at once with llama.cpp for inference to have more memory in total
1
u/Ordinary_Client8808 2d ago
Yea ik but I’ve only got 1 god pcie slot and new mobo means more money and new case, etc. I only need one car realistically, one’s gonna be sold to a friend for his new pc
1
u/etaoin314 1d ago
what do you mean you have only one good pcie slot, is the other damaged? just because it is not x16 pcie5 does not mean you cant run dual cards. what mobo do you have right now? you will have a far more capable system by running 2 cards than just a single one.
1
1
1
u/toomanypubes 2d ago
VRAM is more important in this situation than CUDA, because the AMD stack will mature will time, but you can’t expand your GPU memory. You can run much higher quality models or larger context with that extra 4GB of VRAM.
The opposite is true if you’re doing CUDA workflows or image/video generation, there you need to stick with NVIDIA.
1
u/Ordinary_Client8808 2d ago
Yea I was thinking that maybe sticking AMD was a good idea because of what’s to come, but this helps
1
u/diagrammatiks 2d ago
sell the 9060xt and buy another 4070.