Skip to content

Commit

Permalink
Add ability to add tag to pslots with generate_workflows (NOAA-EMC#3036)
Browse files Browse the repository at this point in the history
# Description
`generate_workflows.sh` would only use the case name as the pslot,
leading to conflicts if you try to run two different sets at the same
time. Now there is a new option that allows a 'tag' to the case name
when determining the pslots:

```
generate_workflows.sh -t <tag>

tag: string to be added to end of case name
```

For example, this:

```
generate_workflows.sh -t test
```

Will result in pslots of `C48_ATM_test`, `C48_S2SW_test`, etc.

This is similar to how the CI system appends the hash to the pslots, but
allows flexibility since this is a command-line tool.

# Type of change
- [ ] Bug fix (fixes something broken)
- [x] New feature (adds functionality)
- [ ] Maintenance (code refactor, clean-up, new CI test, etc.)

# Change characteristics
- Is this a breaking change (a change in existing functionality)? NO
- Does this change require a documentation update? NO
- Does this change require an update to any of the following submodules?
NO

# How has this been tested?
- Ran `generate_workflows.sh` both with and without the new `-t` option.

# Checklist
- [x] Any dependent changes have been merged and published
- [x] My code follows the style guidelines of this project
- [x] I have performed a self-review of my own code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have documented my code, including function, input, and output
descriptions
- [x] My changes generate no new warnings
- [x] New and existing tests pass with my changes
- [ ] This change is covered by an existing CI test or a new one has
been added
- [x] Any new scripts have been added to the .github/CODEOWNERS file
with owners
- [x] I have made corresponding changes to the system documentation if
necessary
  • Loading branch information
WalterKolczynski-NOAA authored Oct 25, 2024
1 parent 58737cb commit 39d0b8b
Showing 1 changed file with 10 additions and 4 deletions.
14 changes: 10 additions & 4 deletions workflow/generate_workflows.sh
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,9 @@ function _usage() {
If this option is not specified, then the existing email address in
the crontab will be preserved.
-t Add a 'tag' to the end of the case names in the pslots to distinguish
pslots between multiple sets of tests.
-v Verbose mode. Prints output of all commands to stdout.
-V Very verbose mode. Passes -v to all commands and prints to stdout.
Expand Down Expand Up @@ -101,6 +104,7 @@ _hpc_account=""
_set_account=false
_update_cron=false
_email=""
_tag=""
_set_email=false
_verbose=false
_very_verbose=false
Expand All @@ -111,7 +115,7 @@ _runtests="${RUNTESTS:-${_runtests:-}}"
_nonflag_option_count=0

while [[ $# -gt 0 && "$1" != "--" ]]; do
while getopts ":H:bB:uy:Y:GESA:ce:vVdh" option; do
while getopts ":H:bB:uy:Y:GESA:ce:t:vVdh" option; do
case "${option}" in
H)
HOMEgfs="${OPTARG}"
Expand All @@ -138,6 +142,7 @@ while [[ $# -gt 0 && "$1" != "--" ]]; do
S) _run_all_sfs=true ;;
c) _update_cron=true ;;
e) _email="${OPTARG}" && _set_email=true ;;
t) _tag="_${OPTARG}" ;;
v) _verbose=true ;;
V) _very_verbose=true && _verbose=true && _verbose_flag="-v" ;;
d) _debug=true && _very_verbose=true && _verbose=true && _verbose_flag="-v" && PS4='${LINENO}: ' ;;
Expand Down Expand Up @@ -454,11 +459,12 @@ echo "Running create_experiment.py for ${#_yaml_list[@]} cases"
[[ "${_verbose}" == true ]] && printf "Selected cases: %s\n\n" "${_yaml_list[*]}"
for _case in "${_yaml_list[@]}"; do
[[ "${_verbose}" == false ]] && echo "${_case}"
_pslot="${_case}${_tag}"
_create_exp_cmd="./create_experiment.py -y ../ci/cases/pr/${_case}.yaml --overwrite"
if [[ "${_verbose}" == true ]]; then
pslot=${_case} RUNTESTS=${_runtests} ${_create_exp_cmd}
pslot=${_pslot} RUNTESTS=${_runtests} ${_create_exp_cmd}
else
if ! pslot=${_case} RUNTESTS=${_runtests} ${_create_exp_cmd} 2> stderr 1> stdout; then
if ! pslot=${_pslot} RUNTESTS=${_runtests} ${_create_exp_cmd} 2> stderr 1> stdout; then
_output=$(cat stdout stderr)
_message="The create_experiment command (${_create_exp_cmd}) failed with a non-zero status. Output:"
_message="${_message}"$'\n'"${_output}"
Expand All @@ -471,7 +477,7 @@ for _case in "${_yaml_list[@]}"; do
fi
rm -f stdout stderr
fi
grep "${_case}" "${_runtests}/EXPDIR/${_case}/${_case}.crontab" >> tests.cron
grep "${_pslot}" "${_runtests}/EXPDIR/${_pslot}/${_pslot}.crontab" >> tests.cron
done
echo

Expand Down

0 comments on commit 39d0b8b

Please sign in to comment.