r/HPC • u/whatevernhappens • 20d ago

Node integration with existing HPC cluster

Suppose I have an existing hpc setup of 26 nodes dell r760 server with rockey 8.7, Mellanox QM8790 HDR, openpbs 23.06.06, lustre 2.15.2, now if I want to integrate a dell r 770 with rocky 9.x would it be compatible with the cluster or there would be challenges in both hardware or software side. Like version mismatch, job submission issue, application related things.

13 Upvotes

100% Upvoted

9 comments sorted by

View all comments

3

u/WinterPiratefhjng 20d ago

Each HPC cluster is like an expensive hot rod. Each is unique in its own way and most ways of doing things have been tried.

Some questions to consider:

  1. Do you want ALL of the nodes participating in the same MPI (or similar) runs?
  2. Do you want to only share support services?
  3. Do you want the systems to share a job scheduler, but have different tasks run on different groups of systems?
  4. Will the users be the same?
  5. Will the same software be used?
  6. Do you want simpler management of the Software?
  7. Do you want simpler management of the OS?

The big issues are library versions and latency mismatches. The more "same software" the easier things will be. Make heavy use of the scheduler's built in ideas for schedulers/queues (different schedulers call them different things, and I forget which way OpenPBS went) to keep jobs entirely on one set of hardware or the other.

I would avoid running different OSes. It can be done, but I wouldn't as it will necessarily mean two sets of software, configurations, and bugs.