sometimes we want to run srun or jsrun with very specific arguments, that are not covered by the standard inputs. For example in SLURM, an mpi job should that uses 3 tasks, but 8 Nodes are in the pool will ask for 8 nodes, but then warns that the number of used nodes is reduced to 3. I want neither 3 nor 8 nodes, but 1. But maybe for memory intensive tasks you would want 3 nodes. It really depends on the application.
I suggest a keyword like optional_slurm_args and optional_lsf_args which is a string that can be append to say stuff like. This
--exclusive -N1 and others and will just be appended to the mpiexec call.
IMPORTANT
I think differentiating between the schedulers for theses kwargs is important to keep portability. So, just having optional_kwargs wouldn't make sense.
What do you think @icui ? Do you have a better idea how to handle corner cases?
sometimes we want to run
srunorjsrunwith very specific arguments, that are not covered by the standard inputs. For example inSLURM, an mpi job should that uses 3 tasks, but 8 Nodes are in the pool will ask for 8 nodes, but then warns that the number of used nodes is reduced to 3. I want neither 3 nor 8 nodes, but 1. But maybe for memory intensive tasks you would want 3 nodes. It really depends on the application.I suggest a keyword like
optional_slurm_argsandoptional_lsf_argswhich is a string that can be append to say stuff like. This--exclusive -N1and others and will just be appended to the mpiexec call.IMPORTANT
I think differentiating between the schedulers for theses
kwargsis important to keep portability. So, just havingoptional_kwargswouldn't make sense.What do you think @icui ? Do you have a better idea how to handle corner cases?