Added more metadata to Dask Dataframe creation (#19)

Fixed #17. It looks like it is, in fact, lazily opened. `len(era5_df)` requires a full scan. I opened #18 to address the length issue.
alxmrs · Feb 19, 2024 · 7da2184 · 7da2184
1 parent cdc5d41
commit 7da2184
Show file tree

Hide file tree

Showing 9 changed files with 1,304 additions and 5 deletions.
diff --git a/perf_tests/open_era5.py b/perf_tests/open_era5.py
@@ -0,0 +1,13 @@
+#!/usr/bin/env python3
+
+import xarray as xr
+import qarray as qr
+
+# Requires authenticating with GCP
+era5_ds = xr.open_zarr(
+  'gs://gcp-public-data-arco-era5/ar/1959-2022-full_37-1h-0p25deg-chunk-1.zarr-v2',
+  chunks={'time': 240, 'level': 1}
+)
+era5_wind_df = qr.to_dd(era5_ds[['u_component_of_wind', 'v_component_of_wind']])
+
+print(era5_wind_df.columns)