Message Logging generates duplicated output lines

[TonyB9000]

Naturally, e3sm_tocmip (E2C) establishes its own “python logging” configuration to track its many operations. However, there are several places in E2C where “cmor.setup()” is called, and (as far as I can tell) the only access to CMOR internal logging is the specification of the logfile.

cmor.setup(inpath=table_path, netcdf_file_action=cmor.CMOR_REPLACE, logfile=logfile)

Somehow, output log messages are duplicated (even triplicated). Typical E2C output:

20240531_154311_683:INFO:e3sm_to_cmip.util:find_mpas_files:component mpas_map found: /p/user_pub/e3sm/staging/resource/maps/map_EC30to60E2r2_to_cmip6_180x360_aave.20220301.nc
2024-05-31 15:43:11,683 [INFO]: util.py(find_mpas_files:554) >> component mpas_map found: /p/user_pub/e3sm/staging/resource/maps/map_EC30to60E2r2_to_cmip6_180x360_aave.20220301.nc
2024-05-31 15:43:11,683 [INFO]: util.py(find_mpas_files:554) >> component mpas_map found: /p/user_pub/e3sm/staging/resource/maps/map_EC30to60E2r2_to_cmip6_180x360_aave.20220301.nc
2024-05-31 15:43:11,684 [INFO]: thkcello.py(handle:47) >> Starting thkcello
2024-05-31 15:43:11,684 [INFO]: thkcello.py(handle:47) >> Starting thkcello
2024-05-31 15:43:11,684 [INFO]: thkcello.py(handle:54) >>   Calling xarray.open_dataset()
2024-05-31 15:43:11,684 [INFO]: thkcello.py(handle:54) >>   Calling xarray.open_dataset()
2024-05-31 15:43:11,817 [INFO]: thkcello.py(handle:63) >>   Calling mpas.open_mfdataset()
2024-05-31 15:43:11,817 [INFO]: thkcello.py(handle:63) >>   Calling mpas.open_mfdataset()
2024-05-31 15:45:37,983 [INFO]: thkcello.py(handle:76) >>   Calling mpas.remap() (mappingFileName=/p/user_pub/e3sm/staging/resource/maps/map_EC30to60E2r2_to_cmip6_180x360_aave.20220301.nc)
2024-05-31 15:45:37,983 [INFO]: thkcello.py(handle:76) >>   Calling mpas.remap() (mappingFileName=/p/user_pub/e3sm/staging/resource/maps/map_EC30to60E2r2_to_cmip6_180x360_aave.20220301.nc)
20240531_154537_993:INFO:e3sm_to_cmip.mpas:remap:Calling ds.transpose
20240531_154537_996:INFO:e3sm_to_cmip.mpas:remap:Calling write_netcdf() (inFileName=/p/user_pub/e3sm/bartoletti1/tmp/tmp2hynl1iz)
20240531_154742_624:INFO:e3sm_to_cmip.mpas:remap:Calling run_ncremap_cmd with args ['ncremap', '-P', 'mpasocean', '-7', '--dfl_lvl=1', '--no_stdin', '--no_cll_msr', '--no_frm_trm', '--no_permute', '--map=/p/user_pub/e3sm/staging/resource/maps/map
_EC30to60E2r2_to_cmip6_180x360_aave.20220301.nc', '/p/user_pub/e3sm/bartoletti1/tmp/tmp2hynl1iz', '/p/user_pub/e3sm/bartoletti1/tmp/tmpcajrkcdp']
2024-05-31 15:49:46,817 [INFO]: thkcello.py(handle:85) >>   Calling mpas.setup_cmor()
2024-05-31 15:49:46,817 [INFO]: thkcello.py(handle:85) >>   Calling mpas.setup_cmor()
2024-05-31 15:49:47,320 [INFO]: thkcello.py(handle:90) >>   Calling mpas.write_cmor()
2024-05-31 15:49:47,320 [INFO]: thkcello.py(handle:90) >>   Calling mpas.write_cmor() 

The logging set-up in E2C employs YYYYMMDD_hhmmss_msec timestamp, and the “lineno” parameter is used nowhere (that I can find, at least) and so I assume this is established in cmor logging.

Is it possible to expose the cmor logging configuration to the calling party, e.g. the “propagate” setting in cmor logging? (not even sure that would help, but would like to try “propagate=False”, or take more control of the stream and file handlers, etc.)