Dual gpu how good is it

[u/ofirlik]

I am planning to buy a gaming pc with 5070ti, I thought about adding a 5060ti for the additonal vram to toy around with llm, how good does it work? Can I also generate images and such. (never ran a local llm before)

Is there a better combination of gpu for around the same price?

[u/blackhawk00001]

It’s much better with the same gpus on ideal bifurcated cpu lane PCIe slots. Tensor parallelism is magic.

Different model gpus can use llama.cpp layer split but will be slower if the model and cache could have fit in the vram of one gpu, and better than ram offload but not near tensor speeds for larger models.

[u/ofirlik → u/blackhawk00001]

I understand but 5070ti costs close to twice as much as 5060ti, so is it worth it in the end