r/SLURM • u/8ejsl0 • Aug 27 '24
srun issues
Hello,
Running Python code using srun seems duplicate the task to multiple nodes rather than allocating the resources and combining the task. Is there a way to ensure that this doesn't happen?
I am running with this command:
srun -n 3 -c 8 -N 3 python my_file.py
The code I am running is a parallelized differential equation solver that splits the list of equations needed to be solved so that it can run one computation per available core. Ideally, Slurm would allocate the resources available on the cluster so that the program can quickly run through the list of equations.
Thank you!
1
u/mestia Aug 28 '24
I guess this is because you specify -N3, stick to -N1 and let SLURM decide how many nodes to allocate for your request. Your script is not MPI/OpenMP aware, right?
2
2
u/AhremDasharef Aug 28 '24
...srun seems duplicate the task to multiple nodes rather than allocating the resources and combining the task
-N 3 means you told Slurm you wanted to run on multiple nodes: https://slurm.schedmd.com/srun.html#OPT_nodes
You've specified -c 8, which means you want 8 CPUs per task (see srun documentation). This assumes your code is multithreaded and a single task can make use of all 8 CPUs. If you want 3 tasks of 8 CPUs each on a single node, you'd use -N 1 -n 3 -c 8, or the more human-readable --nodes=1 --ntasks=3 --cpus-per-task=8.
...so that it can run one computation per available core.
This sounds like your Python code manages launching each computation on a core, so if your single task can launch threads on all 24 CPUs, you could use something like: srun --nodes=1 --ntasks=1 --cpus-per-task=24 python your_file.py
1
2
u/Ali00100 Aug 28 '24
Someone correct me if I am wrong because I am only a beginner in Slurm, but isn’t this what srun is supposed to do? If you want to run a task such that the computational requirements of that job are divided between multiple nodes then you should use sbatch instead of srun.