Visualizing large stacks of results

[scottstanie]

Opening an issue here for possible ways to visualize case studies where we have 10s of GB of data or more, either locally or in an S3 bucket. We're going to have multiple layers (unwrapped phase, coherence, etc.) for each date. We'll usually have somewhere in the 50-500 images over time, with each image being ~5000 x 5000 pixels.

Below is what I've found from browsing around, so please add more options to this as you find them

Nicer lofty goals

• EGMS / Norway web platforms: https://egms.land.copernicus.eu/ https://insar.ngu.no/

  • They show an average velocity as the base raster layer, and give you a time series for any point you click. This seems like the very minimum that we want to do
  • Both are running with the same software. This is not open source.

• The nicest open-source web mapper I've seen for 3-D (or 4-D) raster data is this: https://maps.demo.carbonplan.org/ (repo here https://github.com/carbonplan/maps)

  • This would take some javascript knowledge to get working for our case.
  • Does not appear to have a time series viewer on click, only a slider through time.
  • May not be amenable to ingesting our separate .nc file per date Support for Kerchunk data sources carbonplan/maps#100

Existing simple options

• Just use matplotlib within sweets: https://github.com/isce-framework/sweets/blob/6ee8de6332b92a0475e38efc7afea7e8b56f87c0/src/sweets/plotting.py#L121
• Demo: https://dolphin-insar.readthedocs.io/en/latest/notebooks/walkthrough-basic/#visualization-using-sweets
• This could be adapted to stream data, e.g. using https://github.com/fsspec/s3fs
  • More on using xarray to plot data on S3 https://nasa-openscapes.github.io/2021-Cloud-Hackathon/tutorials/05_Data_Access_Direct_S3.html

Lower hanging fruit which could be improvements for interactivity

• (from Edward Armstrong) https://github.com/podaac/the-coding-club/blob/main/notebooks/SWOT_SSH_dashboard.ipynb this notebook was developed to visualize Level 2 SWOT altimeter data, and you can "drill" to find hi resolution data points. Leverages data in a zarr store (native SWOT data is netCDF) and hvplot
• Using the Holoviews ecosystem with panel, like https://github.com/RaphaelGrandin/FlatsimDashboard
• Some people have liked using https://github.com/opengeos/leafmap . Scott update: I could not make add_raster show up in jupyter for a single image, after 30 minutes of installing things.

Unknown options

• https://xcube.readthedocs.io/en/latest/viewer.html This seems to be doing what we want? At least by the end images. Hard to tell if this is easily usable though
• https://opendatacube.readthedocs.io/en/latest/data-access-analysis/tools-exploring-data/using-juypter.html The "Open data cube" goals also seem aligned with what we want, but also has no easy path to a working example for us

[scottstanie]

closed by bowser, to be released