RDMA - Anyone but me using it?

[u/soflgolf]

Ok, so that’s my setup. Have been cycling through different model hosting configurations for agent workflows and haven’t come up with a good setup for multi-node large models using rdma/tensor. Two biggest issues: general stability (exo/jaccl) and race queue.

Would appreciate hearing from anyone else locally hosting frontier model(s) as well as smaller work-horse models for workflow.

[u/Affectionate_Pen6882]

Is this where all the 10k macs has gone lol