Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge Radmon scripts for monitoring the satellite radiance DA #162

Merged
merged 19 commits into from
Jan 18, 2024
Merged

Merge Radmon scripts for monitoring the satellite radiance DA #162

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
2b4f9b5
Merged the scripts for running the radmon to monitor the satellite ra…
xyzemc Dec 11, 2023
0a7af77
Address the reviewer's coments.
xyzemc Dec 13, 2023
d407efc
Delete fix/gsi/gdas_radmon_base.tar
xyzemc Dec 13, 2023
8a28ed2
modified USHDIR to USHdir
xyzemc Dec 13, 2023
7c6bf37
Change USHnam to USHradmon
xyzemc Dec 13, 2023
c35e456
resove the conflict with dev-sci
xyzemc Dec 18, 2023
0d24581
resove the merge conflict
xyzemc Dec 18, 2023
a19d0f8
Merge branch 'dev-sci' into radmon
xyzemc Dec 18, 2023
30775d2
modified the exnam_verfrad.sh to exrrfs_verfrad.sh
xyzemc Dec 18, 2023
fdef8b8
Merge branch 'dev-sci' into radmon
MatthewPyle-NOAA Dec 19, 2023
2c69010
Removed the local defination of DO_RADMON which is defined in config_…
xyzemc Jan 2, 2024
ed5f609
Merge branch 'radmon' of https://github.com/xyzemc/rrfs-workflow into…
xyzemc Jan 2, 2024
e82911a
Merge branch 'dev-sci' into radmon
chan-hoo Jan 6, 2024
121e915
address reviewer's comments
xyzemc Jan 6, 2024
c560a3b
Merge branch 'dev-sci' into radmon
chan-hoo Jan 16, 2024
bb0ac27
Merge branch 'dev-sci' into radmon
MatthewPyle-NOAA Jan 17, 2024
b8a12ef
Merge branch 'dev-sci' into radmon
MatthewPyle-NOAA Jan 17, 2024
61bf322
Add the option to skip the radmon job when radstat file is not existing.
xyzemc Jan 18, 2024
676cd29
Merge branch 'radmon' of https://github.com/xyzemc/rrfs-workflow into…
xyzemc Jan 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions fix/gsi/gdas_radmon_scaninfo.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
sndrd1_g13 0.00 1.00 65
sndrd2_g13 0.00 1.00 65
sndrd3_g13 0.00 1.00 65
sndrd4_g13 0.00 1.00 65
sndrd1_g14 0.00 1.00 65
sndrd2_g14 0.00 1.00 65
sndrd3_g14 0.00 1.00 65
sndrd4_g14 0.00 1.00 65
sndrd1_g15 0.00 1.00 65
sndrd2_g15 0.00 1.00 65
sndrd3_g15 0.00 1.00 65
sndrd4_g15 0.00 1.00 65
hirs2_n14 -49.50 1.80 56
hirs3_n16 -49.50 1.80 56
hirs3_n17 -49.50 1.80 56
hirs4_n18 -49.50 1.80 56
hirs4_n19 -49.50 1.80 56
amsua_n15 -48.333 3.33 30
amsua_n16 -48.333 3.33 30
amsua_n17 -48.333 3.33 30
amsua_n18 -48.333 3.33 30
amsua_n19 -48.33 3.33 30
amsub_n15 -48.95 1.10 90
amsub_n16 -48.95 1.10 90
amsub_n17 -48.95 1.10 90
atms_npp -52.725 1.11 96
atms_n20 -52.393 1.10 96
avhrr_n18 -52.612 1.182 90
avhrr_metop-a -52.612 1.182 90
avhrr_n19 -52.612 1.182 90
avhrr_metop-b -52.612 1.182 90
avhrr_metop-c -52.612 1.182 90
mhs_n18 -49.444 1.111 90
mhs_n19 -49.444 1.111 90
airs_aqua -48.90 1.10 90
amsua_aqua -48.33 3.33 30
ssmis_f16 0.00 1.00 60
ssmis_f17 0.00 1.00 60
ssmis_f18 0.00 1.00 60
ssmis_f19 0.00 1.00 60
hirs4_metop-a -49.50 1.80 56
hirs4_metop-b -49.50 1.80 56
amsua_metop-a -48.333 3.333 30
amsua_metop-b -48.333 3.333 30
amsua_metop-c -48.333 3.333 30
mhs_metop-a -49.444 1.111 90
mhs_metop-b -49.444 1.111 90
mhs_metop-c -49.444 1.111 90
iasi_metop-a -48.330 3.334 60
iasi_metop-b -48.330 3.334 60
iasi_metop-c -48.330 3.334 60
cris_npp -48.330 3.333 30
cris-fsr_npp -48.330 3.333 30
cris-fsr_n20 -48.330 3.333 30
msu_n14 -47.370 9.474 90
ssu -35.000 10.000 90
hsb -48.950 1.100 90
seviri_m08 0.00 1.00 90
seviri_m09 0.00 1.00 90
seviri_m10 0.00 1.00 90
seviri_m11 0.00 1.00 90
saphir_meghat -42.960 0.666 130
abi_g16 0.00 1.00 90
abi_g17 0.00 1.00 90
abi_g18 0.00 1.00 90
ahi_himawari8 0.00 1.00 90
ahi_himawari9 0.00 1.00 90
atms_n21 -52.393 1.10 96
gmi_gpm 0.00 1.00 221
cris-fsr_n21 -48.130 3.333 30
viirs-m_npp -56.280 1.265 90
viirs-m_j1 -56.280 1.265 90
viirs-m_j2 -56.280 1.265 90
!sat_sis start step nstep
2 changes: 2 additions & 0 deletions fix/gsi/rrfs_radmon_satype.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
amsua_n18 amsua_n19 atms_npp hirs4_n19 amsua_metop-b iasi_metop-b mhs_metop-b mhs_n19

12 changes: 6 additions & 6 deletions scripts/exrrfs_run_analysis.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1057,9 +1057,9 @@ if [ "${DO_GSIDIAG_OFFLINE}" = "FALSE" ]; then
${APRUN} $pgm -o diag_${type}_${string}.${YYYYMMDDHH}.nc4 pe*.${type}_${loop}.nc4 >>$pgmout 2>errfile
export err=$?; err_chk
mv errfile errfile_nc_diag_cat_$type

cp diag_${type}_${string}.${YYYYMMDDHH}.nc4 ${COMOUT}
echo "diag_${type}_${string}.${YYYYMMDDHH}.nc4*" >> listcnv
gzip diag_${type}_${string}.${YYYYMMDDHH}.nc4
cp diag_${type}_${string}.${YYYYMMDDHH}.nc4.gz ${COMOUT}
echo "diag_${type}_${string}.${YYYYMMDDHH}.nc4.gz" >> listcnv
numfile_cnv=`expr ${numfile_cnv} + 1`
fi
done
Expand All @@ -1071,9 +1071,9 @@ if [ "${DO_GSIDIAG_OFFLINE}" = "FALSE" ]; then
${APRUN} $pgm -o diag_${type}_${string}.${YYYYMMDDHH}.nc4 pe*.${type}_${loop}.nc4 >>$pgmout 2>errfile
export err=$?; err_chk
mv errfile errfile_nc_diag_cat_$type

cp diag_${type}_${string}.${YYYYMMDDHH}.nc4 ${COMOUT}
echo "diag_${type}_${string}.${YYYYMMDDHH}.nc4*" >> listrad
gzip diag_${type}_${string}.${YYYYMMDDHH}.nc4
cp diag_${type}_${string}.${YYYYMMDDHH}.nc4.gz ${COMOUT}
echo "diag_${type}_${string}.${YYYYMMDDHH}.nc4.gz" >> listrad
numfile_rad=`expr ${numfile_rad} + 1`
else
echo 'No diag_' ${type} 'exist'
Expand Down
59 changes: 53 additions & 6 deletions scripts/exrrfs_run_gsidiag.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -213,8 +213,9 @@ for loop in $loops; do
export err=$?; err_chk
mv errfile errfile_nc_diag_cat_$type

cp diag_${type}_${string}.${YYYYMMDDHH}.nc4 ${COMOUT}
echo "diag_${type}_${string}.${YYYYMMDDHH}.nc4*" >> listcnv
gzip diag_${type}_${string}.${YYYYMMDDHH}.nc4
cp diag_${type}_${string}.${YYYYMMDDHH}.nc4.gz ${COMOUT}
echo "diag_${type}_${string}.${YYYYMMDDHH}.nc4.gz" >> listcnv
numfile_cnv=`expr ${numfile_cnv} + 1`
fi
done
Expand All @@ -226,8 +227,9 @@ for loop in $loops; do
export err=$?; err_chk
mv errfile errfile_nc_diag_cat_$type

cp diag_${type}_${string}.${YYYYMMDDHH}.nc4 ${COMOUT}
echo "diag_${type}_${string}.${YYYYMMDDHH}.nc4*" >> listrad
gzip diag_${type}_${string}.${YYYYMMDDHH}.nc4
cp diag_${type}_${string}.${YYYYMMDDHH}.nc4.gz ${COMOUT}
echo "diag_${type}_${string}.${YYYYMMDDHH}.nc4.gz" >> listrad
numfile_rad=`expr ${numfile_rad} + 1`
else
echo 'No diag_' ${type} 'exist'
Expand All @@ -247,8 +249,9 @@ for loop in $loops; do
export err=$?; err_chk
mv errfile errfile_nc_diag_cat_$type

cp diag_${type}_${string}.${YYYYMMDDHH}.nc4 ${COMOUT}
echo "diag_${type}_${string}.${YYYYMMDDHH}.nc4*" >> listdbz
gzip diag_${type}_${string}.${YYYYMMDDHH}.nc4
cp diag_${type}_${string}.${YYYYMMDDHH}.nc4.gz ${COMOUT}
echo "diag_${type}_${string}.${YYYYMMDDHH}.nc4.gz" >> listdbz
numfile_dbz=`expr ${numfile_dbz} + 1`
fi
done
Expand Down Expand Up @@ -295,6 +298,50 @@ if [ "${DO_RADDA}" = "TRUE" ]; then
cp ./satbias_pc.out ${COMOUT}/rrfs.${spinup_or_prod_rrfs}.${YYYYMMDDHH}_satbias_pc
fi
fi

fi

#------------------------------------------------------------------------
# set up local dirs and run run radmon to generate radiance monitor data
#------------------------------------------------------------------------
DO_RADMON=TRUE
chan-hoo marked this conversation as resolved.
Show resolved Hide resolved

if [ DO_RADMON ]; then
echo "Run EMC Radmon package to generate daily monitoring data for satellite"

envir=${envir:-prod}
REGIONAL_RR=${REGIONAL_RR:-1}

export TANKverf=${TANKverf:-$NWGES_BASEDIR/radmon}
export TANKverf_rad=$TANKverf/radmon.$PDY


if [ ! -d $TANKverf_rad ]; then
mkdir -p -m 775 $TANKverf_rad
fi


export RAD_AREA=${RAD_AREA:-rgn}
export CYCLE_INTERVAL=${CYCLE_INTERVAL:-1}
export DATA=${analworkdir_conv}/radmon
export TANKverf_radM1=${TANKverf_radM1:-${TANKverf}/radmon.${PDY}}

export GSI_MON_BIN=$EXECDIR
chan-hoo marked this conversation as resolved.
Show resolved Hide resolved
export FIXgdas=$FIX_GSI

export RADMON_SUFFIX=rrfs
export rgnHH=${PDY}${cyc}
export biascr=${biascr:-${SATBIAS_DIR}/${RADMON_SUFFIX}.${CYCLE_TYPE}.${rgnHH}_satbias}
export radstat=${radstat:-${SATBIAS_DIR}/${RADMON_SUFFIX}.${CYCLE_TYPE}.${rgnHH}_radstat}

echo "radstat: $radstat"
echo "biascr: $biascr"

export COMPRESS=gzip

CLEAN_TANKVERF=1

. $USHDIR/rrfs_radmon/exrrfs_verfrad.sh ${PDY} ${cyc}
chan-hoo marked this conversation as resolved.
Show resolved Hide resolved
fi
#
#-----------------------------------------------------------------------
Expand Down
1 change: 1 addition & 0 deletions ush/config_defaults.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2202,6 +2202,7 @@ DO_ENSPOST="FALSE"
DO_ENSINIT="FALSE"
DO_SAVE_DA_OUTPUT="FALSE"
DO_GSIDIAG_OFFLINE="FALSE"
DO_RADMON="FALSE"
DO_ENS_RADDA="FALSE"
DO_ENS_BLENDING="FALSE"
ENS_BLENDING_LENGTHSCALE="960" # (Lx) in kilometers
Expand Down
114 changes: 114 additions & 0 deletions ush/rrfs_radmon/clean_tankdir.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
#!/bin/bash
chan-hoo marked this conversation as resolved.
Show resolved Hide resolved

#------------------------------------------------------------------
#
# clean_tankdir.sh
#
#------------------------------------------------------------------

echo "--> clean_tankdir.sh"
rc=0

area=$1
days_to_keep=$2

#---------------------------------------------------------------
# Algorithm
#
# 1. Build list of $run.${PDY} || radmon.${PDY} sudirectories.
# 2. All subdirectories older than $days_to_keep get deleted.
#---------------------------------------------------------------
if [[ ${area} = 'rgn' ]]; then
subdir="radmon"
else
subdir=${RUN}
fi

dirs=`ls ${TANKverf}`

if [[ ${area} = 'rgn' ]]; then
#-----------------------------------------------
# Determine number of days from $PDATE for all
# directories that start with "radmon". Note
# that this expects the directories to be in
# the format of ${RUN}.yyyymmdd.
#
for dir in $dirs; do
file_name="${dir##*/}"
file="${file_name%.*}"
file_extension="${file_name##*.}"

if [ ${file} = ${subdir} ]; then

#----------------------------------------------
# Use date utility to determine number of days
# from PDATE. Note that time difference is
# calculated in seconds, hence the division
# by the number of seconds/day.
#
days=$(( ($(date --date=${PDY} +%s) - $(date --date=${file_extension} +%s) )/(60*60*24) ))

#----------------------------------------
# rm anything older than $days_to_keep
#
if [ $days -gt ${days_to_keep} ]; then

cycle_dir="${TANKverf}/${dir}"

if [[ -d ${cycle_dir} ]]; then
echo "RM ${cycle_dir}"
rm -rf ${cycle_dir}
fi
fi

fi
done

else

#-----------------------------------------------
# Determine number of days from $PDATE for all
# directories that start with "gdas". Note
# that this expects the directories to be in
# the format of ${RUN}.yyyymmdd.
#
for dir in $dirs; do
file_name="${dir##*/}"
file="${file_name%.*}"
file_extension="${file_name##*.}"

if [ ${file} = ${subdir} ]; then

#----------------------------------------------
# Use date utility to determine number of days
# from PDATE. Note that time difference is
# calculated in seconds, hence the division
# by the number of seconds/day.
#
days=$(( ($(date --date=${PDY} +%s) - $(date --date=${file_extension} +%s) )/(60*60*24) ))

#----------------------------------------
# rm anything older than $days_to_keep
#
if [ $days -gt ${days_to_keep} ]; then
cycles="00 06 12 18"

for cyc in $cycles; do
cycle_dir="${TANKverf}/${dir}/${cyc}/radmon"
if [[ -d ${cycle_dir} ]]; then
echo "RM ${cycle_dir}"
rm -rf ${cycle_dir}
fi
done

fi

fi

done
fi

echo "<-- clean_tankdir.sh"

exit ${rc}

Loading