Replace gfs_cyc with an interval

[WalterKolczynski-NOAA]

Description

To facilitate longer and more flexible GFS cadences, the gfs_cyc variable is replaced with a specified interval. Up front, this is
reflected in a change in the arguments for setup_exp to:

--interval <n_hours>

Where n_hours is the interval (in hours) between gfs forecasts. n_hours must be a multiple of 6. If 0, no gfs will be run (only
gdas; only valid for cycled mode). The default value is 6 (every cycle). (This is a change from current behavior of 24.)

In cycled mode, there is an additional argument to control which cycle will be the first gfs cycle:

--sdate_gfs <YYYYMMDDHH>

The default if not provided is --idate + 6h (first full cycle). This is the same as current behavior when gfs_cyc is 6, but may vary from current behavior for other cadences.

As part of this change, some of the validation of the dates has been added. --edate has also been made optional and defaults to --idate if not provided.

During config.base template-filling, INTERVAL_GFS (renamed from STEP_GFS) is defined as --interval and SDATE_GFS as --sdate_gfs`.

Some changes were necessary to the gfs verification (metp) job, as gfs_cyc was being used downstream by verif-global. That has been removed, and instead workflow will be responsible for only running metp on the correct cycles. This also removes "do nothing" metp tasks that exit immediately, because only the last GFS cycle in a day would actually process verification.

Now, metp has its own cycledef and will (a) always runs at 18z, regardless of whether gfs is running at 18z or not, if the interval is less than 24h; (b) use the same cycledef as gfs if the interval is 24h or greater. This is simpler than trying to determine the last gfs cycle of a day when it could change from day to day. To facilitate this change, support for the
undocumented rocoto dependency tag taskvalid is added, as the metp task needs to know whether the cycle has a gfsarch task or not. metp will trigger on gfsarch completing (as before), or look backwards for the last gfsarch to exist.

Additionally, a couple EE2 issues with the metp job are resolved (even though it is not run in ops):

• verif-global update replaced $CDUMP with $RUN
• $DATAROOT is no longer redefined in the metp job

Depends on NOAA-EMC/EMC_verif-global#137
Resolves #260
Refs #1299

Type of change

• New feature (adds functionality)

Change characteristics

• Is this a breaking change (a change in existing functionality)? YES
• Does this change require a documentation update? YES
• Does this change require an update to any of the following submodules? YES
  • EMC verif-global (Remove gfs_cyc dependency and replace CDUMP EMC_verif-global#137)

How has this been tested?

• Cycled test with 24-h interval on Hercules

Checklist

• Any dependent changes have been merged and published
• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• My changes generate no new warnings
• New and existing tests pass with my changes
• I have made corresponding changes to the documentation if necessary

[WalterKolczynski-NOAA]

More manual testing to come, but ready enough to get eyes on it.

[WalterKolczynski-NOAA]

Ready for review. Also encourage third-party testing in excess of what CI will cover due to the nature of the change. I ran some different multi-day experiments myself, but outside validation would be appreciated.

[WalterKolczynski-NOAA]

Failed because restarts from previous attempt weren't wiped, so forecast picked up in the middle.

[emcbot]

CI Update on Wcoss2 at 10/21/24 01:54:58 PM
============================================
Cloning and Building global-workflow PR: 2928
with PID: 52475 on host: clogin01

[emcbot]

Automated global-workflow Testing Results:

Machine: Wcoss2
Start: Mon Oct 21 14:02:10 UTC 2024 on clogin01
---------------------------------------------------
Build: Completed at 10/21/24 02:43:11 PM
Case setup: Completed for experiment C48_ATM_2b6c45de
Case setup: Skipped for experiment C48mx500_3DVarAOWCDA_2b6c45de
Case setup: Skipped for experiment C48_S2SWA_gefs_2b6c45de
Case setup: Completed for experiment C48_S2SW_2b6c45de
Case setup: Completed for experiment C96_atm3DVar_extended_2b6c45de
Case setup: Skipped for experiment C96_atm3DVar_2b6c45de
Case setup: Completed for experiment C96C48_hybatmaerosnowDA_2b6c45de
Case setup: Completed for experiment C96C48_hybatmDA_2b6c45de
Case setup: Completed for experiment C96C48_ufs_hybatmDA_2b6c45de
Case setup: Skipped for experiment C96_S2SWA_gefs_replay_ics_2b6c45de

[WalterKolczynski-NOAA]

All manual CI tests passed on WCOSS:

08:05:32 EXPDIR/>while [[ 1 ]]; do echo; echo; date; for d in C*; do echo "******** ${d} ********"; cd $d; rocotostat -d ${d}.db -w ${d}.xml | tail -n 1; echo; cd ..; done; echo; sleep 300; done


Tue Oct 22 08:05:39 UTC 2024
******** C48_ATM ********
202103231800            gfs_metppcp1                   158916037           SUCCEEDED                   0         1          15.0

******** C48_S2SW ********
202103231800            gfs_metppcp1                   158916989           SUCCEEDED                   0         1          16.0

******** C96_atm3DVar_extended ********
202112211800             gfs_cleanup                   158951431           SUCCEEDED                   0         1          17.0

******** C96C48_hybatmaerosnowDA ********
202112210000        enkfgdas_cleanup                   158920982           SUCCEEDED                   0         1          15.0

******** C96C48_hybatmDA ********
202112210600        enkfgdas_cleanup                   158921341           SUCCEEDED                   0         1          16.0

******** C96C48_ufs_hybatmDA ********
202402240600        enkfgdas_cleanup                   158925325           SUCCEEDED                   0         1          15.0

[DavidHuber-NOAA]

Thanks for all the work on this @WalterKolczynski-NOAA!

[KateFriedman-NOAA]

Looks good, thanks @WalterKolczynski-NOAA !

[DavidHuber-NOAA]

Just noticed that date2_gfs should be date2. I'll fix this in #2943.

[DavidHuber-NOAA]

I'm surprised PEP8 didn't complain about this.