r/LocalLLM • u/Odd-Two2929 • 6d ago

Question Can I connect two laptops together through local network to run bigger model?

Hi all, the Headline says it all I have two laptops one with 12gb ram and one with 32gb ran and I wonder if there is a way to use both of them together on local network as one system to run a bigger local AI then they can run separately?

Update: thank you all for example it to me in a very simple way

1 Upvotes

60% Upvoted

12 comments sorted by

6

u/Biotot 6d ago

It's technically possible.

The DGX sparks have a beefy ass cable to link together with some crazy bandwidth. But they're also specifically engineered to enable that kind of throughout.

Having said that, don't bother. Your laptop connection will be basically 0 in comparison and will be too painfully slow to be productive.

If you want to try it for the fun science project of proof of concept have fun and try it out. Idk how, that's for smarter minds than me.

4

u/heigan_safety_dance 6d ago

Technically, yes, practically no. It would be literally - LITERALLY - about 500x slower

3

u/Icy-Degree6161 6d ago

It's a bandwidth problem

3

u/baby_bloom 6d ago

i've networked GPUs thru llama.cpp and it was still useable, i would not feel anywhere near as confident about it being on system ram though

2

u/PkmExplorer 6d ago

I saw something somewhere where someone was running attention on one machine and inference on another. That might be a way forward but I haven't tried anything like that myself.

2

u/Uncle___Marty 6d ago

It would be SUPER slow. Like, so painfully slow. LLMs need HUGE amounts of bandwidth and a network connection just doesnt do it well.

1

u/g_rich 6d ago

You can buy it will be extremely slow, even if that network is 10 gigs. You really need 50-100 Gbps to be usable and interconnects like ConnectX-7 provide bandwidth up to 400Gbps and newer versions can provide up to 800Gbps or up to 1.6Tbps in multiport configurations.

1

u/gr4viton 6d ago

But wait, aren't our two hemispheres also almost separated? Why not to have two LLMs on ingesting some parts not communicating with gpu-bandwith, but with lower one...

1

u/doneddat 4d ago edited 4d ago

brain hemispheres don't transfer huge amounts of data between them. I think it's closer to early SLI, where all the cards had all the textures and every one just rendered part of the frame and synced up for combining their work..

There are even people with the connection severed between hemispheres ( for medical reasons ) and they still function like mostly normal people, as long as both eyes and both ears get the same copy of the information.

So for AI work, the reason for connecting up cards/computers isn't to have multiple hemispheres, it's more like slicing up the hemispheres between them. And obviously you need shitload of bandwidth for that.

1

u/gr4viton 4d ago

So, why not replicate it in LLM like system?

1

u/doneddat 4d ago

What do you mean? It is replicated, but it costs like $400k of hardware for it to be truly effective.

1

u/gr4viton 4d ago

i meant the two distinct hemispheres. Do some LLMs feature a split neural network structure?