Short description of the diagnostic
As discussed at the recent hackathon, we are aiming for a replacement of diag_scripts/perfmetrics in python. We might be able to use preprocessor functions to calculate different performance metrics in the future. Alternative diagnostics to plot timeseries and hovmöller diagrams already exist. Therefore, we focus on developing a flexible plotting diagnostic that can produce figures similar to the permetrics overview figure 5 and gleckler plots produced by the ETCCDI recipe.
This would be one step towards recreating one figure from the set of perfmetric diagnostics.

• discuss translation of ncl code -> recreate portrait plot from scratch in python
• calculate rmse and pearson corelation using preprocessor
  • Add preprocessors distance_metrics and histogram ESMValCore#2299
• create example plots for some models and variables
• custom y and x axis (metadata keys)
• group into seperate blocks (gaps) (grouby, metadata keys)
• split rectangle into parts (splitby, metadata keys)
  • metadata keys or extra facets for custom labels
  • 2 to 4 triangles per cell. Shape selected based on number of splits
• improve colorbar: layout with figure, customize ticks, labels etc..
• use same scale of colormap for each group plot (does not consider splits)
• customize plot (title, axis labels, cmap etc..)
• custom color for nan values and option to hide nan triangles
• documentation
  • API documentation for diagnostic
  • Recipe documentation (@lukruh)
  • link from/to old/new diags
• example recipe, test recipe (@diegokam, @lukruh )
• test with icon data
• normalization of metrics (mean, median, centered_mean, centered_median)
• add column for mean/median model? (@diegokam)
• recreate perfmetrics example and validate results (@diegokam, @lukruh)
• implement MMM for each group

If we manage to implement index calculation (i.e. via xclim) this could also recreate the figure from the ETCCDI recipe.

Branch and pull request
PR: #3551
Branch: perfmetric_python
Recipe: recipe_perfmetrics_python.yml

Produced figures