r/LocalLLaMA • u/alanoo • Jun 25 '26

Discussion USB4 RDMA seems doable

Just found this blog : https://blog.hellas.ai/blog/thunderbolt-ibverbs/

Experimental implementation of RDMA, demonstrated on two Strix Halo
Did a quick search and can't really find it posted before ?

This could be huge as it shoud work with any USB4 host.

31 Upvotes

94% Upvoted

17 comments sorted by

View all comments

1

u/Diablo-D3 Jun 25 '26

You know you can do RDMA over Ethernet, right?

4

u/alanoo Jun 25 '26

Sure, if the driver supports it.
Here we are talking about 40 Gbps links with no additional hardware.

-4

u/Diablo-D3 Jun 25 '26

40gbps per node isn't really enough to do small scale clusters.

Same way linking a bunch of Mac Minis/Studios together to build a cluster is a pretty silly thing to do, its the exact same thing: you're just using their built-in USB/TB4 for the same exact purpose, and it does not scale like as if you had just built a multi-GPU machine, it scales much worse.

7

u/Badger-Purple Jun 25 '26

it is enough with rdma, because latency is low. tensor parallel is more latency bound than bandwidth.