2024-07-01 Pyaerocom Meeting

[lewisblake]

Last meeting

• 2024-06-24 Pyaerocom Meeting #1222

Next milestone: m2024-07

PRs

• MDA8 #1224

Other Issues

Non-exhaustive list of issues we should assign a milestone to:

• Replacement for the geojsoncontour package needs to be found #952
• add test for working aeroval diurnal cycle analysis to CI #926
• add units systematically when mean and stdev are provided #1021
• GHOST data update #1086
• Allow configuration and implementation of custom statistics in calc_statistics() #1114
• use_obs_clim is broken and has always been: Turns all obsdata into nan #1125
• doubled keys in json config file #1173

AOB

• AeroTools Backend Design Retreat 13.08.24. Idea: Create a label: "design retreat" or similar, and label issues related to topics we would like to address so that we can decide on different scopes which individuals can focus on prior to and present on during the retreat.

[lewisblake]

Topics:

• Configuration files: Options, SOTA, pros and cons of each, backwards compatibility, rerunning experiments.
  • @heikoklein: Guideline on config-files: pyaerocom uses currently .ini, .toml, .yml, .py and .json as config-files (maybe more). This should be unified and a guideline should be given.
• Caching / intermediate representation of data sets
  • Pyaerocom's current appraoch of reading all the raw data all the time
• Pyaerocom vs pyaeroval. What needs to be done to split.
• Model readers. Can something similar to pyaro be done for model readers?
  • Define an interface that a colocator expects from a model reader. Then this interface can be expanded to define different types of colocation (e.g., non-lon lat grids, station colocator)
• Design of GriddedData, UngriddedData, and ColocatedData
• Outputs to the aerovalDB
• Performance: memory consumption, parallelization, edge computing
• General workflow for processing