You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently pre_processing_stage and post_processing_stage are launched in single process/threaded mode, execution_stage is ran in mpi mode through "srun --unbuffered --mpi=pmi2".
In some cases model developer would require more flexibility to control how each step is launched.
We could allow developer to specify a string/command in the manifest.json for each step and insert the string/command between and .
This will be a optional switch, and only override old sbatch file if they are present
for example, this following manifest will allow all the 3 steps to be launched by MPI
The text was updated successfully, but these errors were encountered:
zhiyuli
changed the title
Allow model developer to control launching method for pre_processing_stage, execution_stage and post_processing_stage
[feature request]Allow model developer to control launching method for pre_processing_stage, execution_stage and post_processing_stage
Nov 5, 2021
Currently pre_processing_stage and post_processing_stage are launched in single process/threaded mode, execution_stage is ran in mpi mode through "srun --unbuffered --mpi=pmi2".
In some cases model developer would require more flexibility to control how each step is launched.
We could allow developer to specify a string/command in the manifest.json for each step and insert the string/command between and .
This will be a optional switch, and only override old sbatch file if they are present
for example, this following manifest will allow all the 3 steps to be launched by MPI
{
"name": "MPI Hello World",
"container": "mpich",
"pre_processing_stage": "python setup.py",
"pre_processing_stage_launching_command": "xxxxxxxxxx1",
"execution_stage_launching_command": "xxxxxxxxxxxxxxx2",
"post_processing_stage_launching_command": "xxxxxxxxx3",
}
The text was updated successfully, but these errors were encountered: