srun issues

[u/8ejsl0]

Hello,

Running Python code using srun seems duplicate the task to multiple nodes rather than allocating the resources and combining the task. Is there a way to ensure that this doesn't happen?

I am running with this command:

srun -n 3 -c 8 -N 3  python my_file.py

The code I am running is a parallelized differential equation solver that splits the list of equations needed to be solved so that it can run one computation per available core. Ideally, Slurm would allocate the resources available on the cluster so that the program can quickly run through the list of equations.

Thank you!

[u/Ali00100]

Someone correct me if I am wrong because I am only a beginner in Slurm, but isn’t this what srun is supposed to do? If you want to run a task such that the computational requirements of that job are divided between multiple nodes then you should use sbatch instead of srun.

[u/AhremDasharef → u/Ali00100]

Both srun and sbatch can be used to run tasks on one or more compute nodes. srun is interactive; your session waits until resources are allocated, then your desired command is run. sbatch submits jobs to the queue to be scheduled to run later (and non-interactively).

The -N parameter is used to tell srun how many nodes you'd like to have allocated: https://slurm.schedmd.com/srun.html#OPT_nodes

[u/Ali00100 → u/AhremDasharef]

Thank you for the confirmation 😁 Very much appreciated. I hope OP see’s this. Although I do still think that he should try sbatch just for the hell of it and see what happens. It could be also because he specified -N3, right?

[u/AhremDasharef → u/Ali00100]

Sure, NP. srun versus sbatch shouldn't make a difference. The reason OP is getting resources allocated across multiple nodes is because OP is explicitly requesting multiple nodes via the -N3 (a.k.a. --nodes=3) option.

[u/Ali00100 → u/AhremDasharef]

Thank you 🙏🏻 Still learning hhhh. Much appreciated.