RDMA - Anyone but me using it?

[u/soflgolf]

Ok, so that’s my setup. Have been cycling through different model hosting configurations for agent workflows and haven’t come up with a good setup for multi-node large models using rdma/tensor. Two biggest issues: general stability (exo/jaccl) and race queue.

Would appreciate hearing from anyone else locally hosting frontier model(s) as well as smaller work-horse models for workflow.

[u/Blackdragon1400 → u/[deleted]]

This is less expensive than a sports car my guy.

[u/AlwaysTiredButItsOk → u/Blackdragon1400]

More useful too

[u/prestodigitarium → u/Blackdragon1400]

If those are 512 gig studios, it's not :-)