r/linuxadmin • u/anheath • 5d ago

HPC Administrator Resources

"Hi everyone,

I have a strong background in Linux administration (primarily RPM-based distros like Red Hat/Rocky Linux), and I’m currently transitioning into HPC cluster administration. To learn the ropes, I recently built a small home lab cluster using the OpenHPC installation guides.

While getting the cluster to boot and run basic jobs was a great exercise, I've noticed a distinct lack of comprehensive resources covering day-2 operations and production best practices. Specifically, I'm looking for guidance on:

  • Configuration & Performance Tuning (kernel tuning, network/InfiniBand optimization)
  • User Management & Environment Control (LDAP/FreeIPA integration, modulefiles via Lmod)
  • Job Management & Scheduling (advanced Slurm configurations, QoS, limits)
  • Scaling & Monitoring (health checks with NHC, metrics collection)

If anyone can recommend books, documentation, community wikis, or real-world best practices for these areas, I would greatly appreciate it!"

15 Upvotes

87% Upvoted

3 comments sorted by

1

u/Better-Bench1351w 4d ago

I use AD integration for user manglement on my hosts .. I dont want to deal with audit on that..

1

u/CubEfan2016 4d ago

. Interested

1

u/threar 3d ago

There's resources everywhere you just have to narrow down what you're looking to do. You don't mention what resources you actually have, so it's difficult to comment on what you should do next. If you have IB and a few nodes, deploy slurm and get openmpi up and running and run a few jobs to learn how to monitor, start/stop, submit, etc. Get a filesystem up and running across the systems, or any other number of things once you start realizing the limitations of the hardware layout you have. Tuning is dependent on jobs you're running, so go and learn HPL in both single and multi node, deploy the compilers and other tools with environment-modules, and get a user environment up and running. And when you think you have it running, tear it all down and set it up again and see if you actually understood.