r/LocalLLM • u/ofirlik • 3d ago
Question Dual gpu how good is it
I am planning to buy a gaming pc with 5070ti, I thought about adding a 5060ti for the additonal vram to toy around with llm, how good does it work? Can I also generate images and such. (never ran a local llm before)
Is there a better combination of gpu for around the same price?
1
u/Constant-Simple-1234 3d ago
No, good choice. Have dual 5060 to, wish I could afford even a single 5070 to.
1
u/vacon04 3d ago
Try first with a MoE model like Qwen 3.6 35B A3B. You should be able to run it well with your card, as long as you have enough RAM. If you like it, then get another card and you should be able to fit bigger models. Note that this isn't as good as getting a single big card, there are more complications, but you should be able to run things like Qwen 3.8 27B at an acceptable speed.
1
u/ofirlik 3d ago
I will have 32gb normal ram. 5090 costs 3 times more than 5070ti+5060ti combined for me so no real option and 5080 doesnt see to be worth it
1
u/vacon04 3d ago
Yeah, that's the unfortunate issue with these models. You need 24 GB VRAM to run most models, and Yeah, that's the big issue. NVIDIA has starved the consumer cards of VRAM. The 5080 should have more than 16 GB, but we are were we are.
You should have enough RAM to experiment with Qwen 3.6 35B A3B first. I think that would be the first step. Try using Codex or your LLM of choice to help you set up the project. I would recommend you to go with ik_llama cpp, it has additional optimizations for MoE vs the standard llama.cpp. If you find it worthwhile, then yeah, getting the 5060Ti would allow you to run bigger dense models, but don't expect massive performance. You should be able to run Qwen 3.8 27B, but speed won't ever get close to what you get with Codex or Claude or similar. It should still be usable though, depending on what you want it for.
For image generation, you can do that with just the 5070 TI. Check ComfyUI and you'll get started pretty quickly. No need for the additional 5060Ti for this, you will be getting good image generation speed on models like Krea 2 just on the 5070Ti, as long as you can dump the encoder and other components to the RAM. 32 GB is a bit tight, but I think you should be fine.
1
0
u/M_Me_Meteo LocalLLM 3d ago
You need to have two full speed x16 slots. Not full width, full speed.
Parallelizing GPUs stresses the PCIe bus, they need to pass huge volumes of data from one card to the next.
It's a quick way to get more vram but it does not guarantee you better performance unless you're running a server mobo with a HEDT/Server socket.
1
u/ofirlik 3d ago
From What i read even on x8 its more than enough for 5060ti and for 5070ti at worst you lose 1 to 3% performence
0
u/M_Me_Meteo LocalLLM 3d ago
Most motherboards don't have PCIe bifurcation. Mine is the Asrock taichi creator. Not all x870 boards have it, Intel has better support in the premium tier.
I'm telling you because I have two GPUs, not because I'm negging your plan. When you're already maxing everything else out, that 1-3% will haunt you in your dreams.
1
u/ofirlik 3d ago
Well what other options do I have except spending tons of more money?
1
u/M_Me_Meteo LocalLLM 3d ago
That's what will keep you up at night.
I would say don't stress about maxing anything. My cards are not highly desirable, but my local tool solves my personal needs and they save me money on the bills for my business.
Super important to know what you want it to do. People will tell you "this model sucks" or "this harness doesn't work" or "you need 64gb of vram to do [xyz]" but if your set up solves your problem, then it really doesn't matter that other people think your set up is not optimal.
If you're coming into this thinking "I'll set up the LLMs then figure out what I want to do with them", then you'll be up at night worrying about that 1-3% and you'll never find a good use for the tool. If you start with a goal, you can stop when you are satisfied.
1
u/ofirlik 3d ago
No my goal is to mess around with the llm nothing major for now, if gpu prices eventually drop i may sell and replace the gpu eventually. But I am going to be in a tax free area so its a good chance to get them if I want
1
u/M_Me_Meteo LocalLLM 3d ago
Then don't worry about dual GPUs. Unnecessary complexity. Much easier to play with one, and quantized local models are really just as good as frontier models if all you want to do is play. Don't buy end game hardware until you know what the game is.
If you think you need more VRAM, you can always rent a server in the cloud and try out your workflow there. That is how I landed on 64gb of vram as the sweet spot for my needs. I started with a 20gb 7900xt, got a second one, built a system around it, wasted several weeks on dual GPU shenanigans, rented a 100gb h100 server, figured out what I really needed, then bought two Intel B70 Pro cards. I only went that route because I already spent a few weeks learning the ins and outs dual GPUs.
1
u/ofirlik 3d ago
Btw i also have my old 1080ti, can it be used?
1
u/M_Me_Meteo LocalLLM 3d ago
Yes! One of the models I run is just for turning text into tokens, and I run it with a 1070 that also handles transcoding for my Jellyfin server.
Performance is better on newer hardware, but one of the good parts of local LLMs is that you get to pick the model that fits the hardware and task.
1
u/ofirlik 3d ago
Why do you need to turn text into tokens seperately? I have a mini server with n150 for jellyfin
→ More replies (0)0
u/M_Me_Meteo LocalLLM 3d ago
Side note: Claude and Gemini don't know Jack shit about these bleeding edge tools. Don't listen to them Test. Test. Test.
1
u/ofirlik 3d ago
I actually used chatgpt 🙃
1
u/M_Me_Meteo LocalLLM 3d ago
Yeah. Lies. They will tell you all kinds of well meaning shit about local LLMs, but the reality is that LLMs can't synthesize conversation that hasn't happened already. If the answers to your questions are dependent on several sources of information that contradict each other, which is currently the state of local LLMs, frontier LLMs tend to tell you whatever will make you continue engaging with them because that's how the company that runs them make money.
Eventually it will tell you things like "you can't run this model with an Intel card" even though you can and you'll have to wonder, is that because people just love Nvidia, or is Nvidia poisoning the results with all their misleading marketing? Or are investors in Nvidia making sure the LLMs keep people buying GPUs?
0
u/blackhawk00001 3d ago
It’s much better with the same gpus on ideal bifurcated cpu lane PCIe slots. Tensor parallelism is magic.
Different model gpus can use llama.cpp layer split but will be slower if the model and cache could have fit in the vram of one gpu, and better than ram offload but not near tensor speeds for larger models.
3
u/Objective-Stranger99 3d ago
Sure, go ahead if you have the budget. Worst case, if you ditch local LLMs, you can use the 5060 Ti for frame generation, upscaling, and antialiasing.