remove cuda-nvcc and document ptxas (pangeo-data#398)

* remove cudanvcc
* [condalock-command] autogenerated conda-lock files
* add note about ptxas

Co-authored-by: Wei Ji <[email protected]>
Co-authored-by: pangeo-bot <[email protected]>
Co-authored-by: Wei Ji <[email protected]>

README.md

@@ -170,6 +170,7 @@ The runtime environment sets two variables by default
 * Since 2020.10.16, [mamba](https://github.com/mamba-org/mamba) is installed into the base-image and conda-lock environment and is used by default to solve for a compatible environment (see #146)
 * For a simple list of packages for a given image, you can use a link like this: https://github.com/pangeo-data/pangeo-docker-images/blob/2020.10.08/pangeo-notebook/packages.txt
 * To compare changes between two images, you can use a link like this: https://github.com/pangeo-data/pangeo-docker-images/compare/2020.10.03..2020.10.08
+* Our `ml-notebook` image now contains JAX and TensorFlow with XLA enabled. Due to licensing issues, conda-forge does not have `ptxas`, but `ptxas` is needed for XLA to work correctly. Should you like to use JAX and/or TensorFlow with XLA optimization, please install `ptxas` on your own, for example, by `conda install -c nvidia cuda-nvcc`. At the time of writing (October 2022), JAX throws a compilation error if the `ptxas` version is higher than the driver version. There does not exist an easy solution for K80 GPUs, but in the case of T4 GPUs, you should install `conda install -c nvidia cuda-nvcc==11.6.*` to be safe. Alternatively for any GPU, you could set an environment variable to resolve the error caused by JAX: `XLA_FLAGS="--xla_gpu_force_compilation_parallelism=1"`. The aforementioned error will be removed (and likely turned into a warning) in a future version of JAX. See https://github.com/google/jax/issues/12776#issuecomment-1276649134
 
 
 ### Dask-gateway compatibility

base-notebook/conda-linux-64.lock

@@ -131,7 +131,7 @@ https://conda.anaconda.org/conda-forge/linux-64/markupsafe-2.1.1-py39hb9d737c_1.
 https://conda.anaconda.org/conda-forge/noarch/matplotlib-inline-0.1.6-pyhd8ed1ab_0.tar.bz2#b21613793fcc81d944c76c9f2864a7de
 https://conda.anaconda.org/conda-forge/linux-64/msgpack-python-1.0.4-py39hf939315_0.tar.bz2#35b4a1a56408657cd2c6ce7145c21ecf
 https://conda.anaconda.org/conda-forge/linux-64/multidict-6.0.2-py39hb9d737c_1.tar.bz2#f13f8563cbabc9e3ed69037257124f5d
-https://conda.anaconda.org/conda-forge/linux-64/numpy-1.23.3-py39hba7629e_0.tar.bz2#320e25179733ec4a2ecffcebc8abbc80
+https://conda.anaconda.org/conda-forge/linux-64/numpy-1.23.4-py39h3d75532_0.tar.bz2#ed1301e6c4c15a6419e2d2abbf6a7694
 https://conda.anaconda.org/conda-forge/noarch/packaging-21.3-pyhd8ed1ab_0.tar.bz2#71f1ab2de48613876becddd496371c85
 https://conda.anaconda.org/conda-forge/noarch/partd-1.3.0-pyhd8ed1ab_0.tar.bz2#af8c82d121e63082926062d61d9abb54
 https://conda.anaconda.org/conda-forge/noarch/pexpect-4.8.0-pyh9f0ad1d_2.tar.bz2#5909e7b978141dd80d28dbf9de627827
@@ -145,7 +145,7 @@ https://conda.anaconda.org/conda-forge/noarch/python-dateutil-2.8.2-pyhd8ed1ab_0
 https://conda.anaconda.org/conda-forge/linux-64/pyyaml-6.0-py39hb9d737c_4.tar.bz2#dcc47a3b751508507183d17e569805e5
 https://conda.anaconda.org/conda-forge/linux-64/pyzmq-24.0.1-py39headdf64_0.tar.bz2#5f8db6d0d93052050562f0aecac242c5
 https://conda.anaconda.org/conda-forge/linux-64/ruamel.yaml.clib-0.2.6-py39hb9d737c_1.tar.bz2#a0fabd69dd35bb24ec84d28dc01c3c5b
-https://conda.anaconda.org/conda-forge/noarch/tinycss2-1.1.1-pyhd8ed1ab_0.tar.bz2#5d280406501e79dc7aa9c9ac31d25a80
+https://conda.anaconda.org/conda-forge/noarch/tinycss2-1.2.1-pyhd8ed1ab_0.tar.bz2#7234c9eefff659501cd2fe0d2ede4d48
 https://conda.anaconda.org/conda-forge/linux-64/tornado-6.1-py39hb9d737c_3.tar.bz2#5e13a2d214ed4184969df363a1aab420
 https://conda.anaconda.org/conda-forge/noarch/typing-extensions-4.4.0-hd8ed1ab_0.tar.bz2#be969210b61b897775a0de63cd9e9026
 https://conda.anaconda.org/conda-forge/noarch/zict-2.2.0-pyhd8ed1ab_0.tar.bz2#cd563d01df94e51f968645dbf3b310b0
@@ -175,7 +175,7 @@ https://conda.anaconda.org/conda-forge/noarch/argon2-cffi-21.3.0-pyhd8ed1ab_0.ta
 https://conda.anaconda.org/conda-forge/noarch/bokeh-2.4.3-pyhd8ed1ab_3.tar.bz2#e4c6e6d99add99cede5328d811cacb21
 https://conda.anaconda.org/conda-forge/noarch/jupyter_telemetry-0.1.0-pyhd8ed1ab_1.tar.bz2#bb9ebdb6d5aa2622484aff1faceee181
 https://conda.anaconda.org/conda-forge/noarch/nbformat-5.7.0-pyhd8ed1ab_0.tar.bz2#f8439ce65664bec6d348346590ed57f0
-https://conda.anaconda.org/conda-forge/noarch/oauthlib-3.2.1-pyhd8ed1ab_0.tar.bz2#0059a49c6f1410aab3c3b5572feb1a41
+https://conda.anaconda.org/conda-forge/noarch/oauthlib-3.2.2-pyhd8ed1ab_0.tar.bz2#8f882b197fd9c4941a787926baea4868
 https://conda.anaconda.org/conda-forge/noarch/prompt-toolkit-3.0.31-pyha770c72_0.tar.bz2#94fee21afe199936bc361608e537cd11
 https://conda.anaconda.org/conda-forge/noarch/pyopenssl-22.1.0-pyhd8ed1ab_0.tar.bz2#fbfa0a180d48c800f922a10a114a8632
 https://conda.anaconda.org/conda-forge/noarch/certipy-0.1.3-py_0.tar.bz2#23486713ef5712923e7c57cae609b22e
@@ -195,7 +195,7 @@ https://conda.anaconda.org/conda-forge/noarch/dask-gateway-2022.10.0-pyh8af1aa0_
 https://conda.anaconda.org/conda-forge/noarch/jupyter-server-proxy-3.2.2-pyhd8ed1ab_0.tar.bz2#77b213af8a32bdc5b25c0c3fde58f889
 https://conda.anaconda.org/conda-forge/noarch/jupyterlab_server-2.16.0-pyhd8ed1ab_0.tar.bz2#49468b3799f2e2eb3f6ffec92f22c325
 https://conda.anaconda.org/conda-forge/noarch/nbconvert-7.2.1-pyhd8ed1ab_0.tar.bz2#8e12f153954596420bab7cdeeabb3f15
-https://conda.anaconda.org/conda-forge/noarch/notebook-shim-0.1.0-pyhd8ed1ab_0.tar.bz2#3a8e2c7dcc674f2cb0784f1faba57055
+https://conda.anaconda.org/conda-forge/noarch/notebook-shim-0.2.0-pyhd8ed1ab_0.tar.bz2#45b287252fd17eccb002676623ba859c
 https://conda.anaconda.org/conda-forge/noarch/nbclassic-0.4.5-pyhd8ed1ab_0.tar.bz2#8d16bf3ba8cba7f702691767919d6bc6
 https://conda.anaconda.org/conda-forge/noarch/pangeo-dask-2022.10.17-hd8ed1ab_0.tar.bz2#5ffcf76c840b4546dd20d7385249dc63
 https://conda.anaconda.org/conda-forge/noarch/notebook-6.5.1-pyha770c72_0.tar.bz2#16d5690e1f1cf716b6adf33a2a54dce5

base-notebook/packages.txt

@@ -121,9 +121,9 @@ nbgitpuller==1.1.0
 ncurses==6.3
 nest-asyncio==1.5.6
 notebook==6.5.1
-notebook-shim==0.1.0
-numpy==1.23.3
-oauthlib==3.2.1
+notebook-shim==0.2.0
+numpy==1.23.4
+oauthlib==3.2.2
 openjpeg==2.5.0
 openssl==1.1.1q
 packaging==21.3
@@ -178,7 +178,7 @@ sqlite==3.39.4
 stack_data==0.5.1
 tblib==1.7.0
 terminado==0.16.0
-tinycss2==1.1.1
+tinycss2==1.2.1
 tk==8.6.12
 tomli==2.0.1
 toolz==0.12.0

forge/conda-linux-64.lock

@@ -8,7 +8,7 @@ https://conda.anaconda.org/conda-forge/noarch/font-ttf-dejavu-sans-mono-2.37-hab
 https://conda.anaconda.org/conda-forge/noarch/font-ttf-inconsolata-3.000-h77eed37_0.tar.bz2#34893075a5c9e55cdafac56607368fc6
 https://conda.anaconda.org/conda-forge/noarch/font-ttf-source-code-pro-2.038-h77eed37_0.tar.bz2#4d59c254e01d9cde7957100457e2d5fb
 https://conda.anaconda.org/conda-forge/noarch/font-ttf-ubuntu-0.83-hab24e00_0.tar.bz2#19410c3df09dfb12d1206132a1d357c5
-https://conda.anaconda.org/conda-forge/linux-64/gh-2.17.0-ha8f183a_0.tar.bz2#b7394e6c68120984cddd36493963c224
+https://conda.anaconda.org/conda-forge/linux-64/gh-2.18.0-ha8f183a_0.tar.bz2#bddf0a310227510bcc307d71801e44ee
 https://conda.anaconda.org/conda-forge/linux-64/git-lfs-3.2.0-ha770c72_0.tar.bz2#4915c8a17ab5f4477a3c281d76008182
 https://conda.anaconda.org/conda-forge/noarch/kernel-headers_linux-64-2.6.32-he073ed8_15.tar.bz2#5dd5127afd710f91f6a75821bac0a4f0
 https://conda.anaconda.org/conda-forge/linux-64/ld_impl_linux-64-2.39-hc81fddc_0.tar.bz2#c2719e2faa7bd7076d3a4b52271e5622
@@ -366,7 +366,7 @@ https://conda.anaconda.org/conda-forge/noarch/rsa-4.7.2-pyh44b312d_0.tar.bz2#345
 https://conda.anaconda.org/conda-forge/linux-64/rtree-1.0.1-py39hb102c33_0.tar.bz2#166cd8ffc125784089e5fdc1feb4b2e5
 https://conda.anaconda.org/conda-forge/linux-64/ruamel.yaml.clib-0.2.6-py39hb9d737c_1.tar.bz2#a0fabd69dd35bb24ec84d28dc01c3c5b
 https://conda.anaconda.org/conda-forge/linux-64/tiledb-2.11.3-h1e4a385_0.tar.bz2#17010e2e381defa8a697b6f00314e6bb
-https://conda.anaconda.org/conda-forge/noarch/tinycss2-1.1.1-pyhd8ed1ab_0.tar.bz2#5d280406501e79dc7aa9c9ac31d25a80
+https://conda.anaconda.org/conda-forge/noarch/tinycss2-1.2.1-pyhd8ed1ab_0.tar.bz2#7234c9eefff659501cd2fe0d2ede4d48
 https://conda.anaconda.org/conda-forge/linux-64/tornado-6.1-py39hb9d737c_3.tar.bz2#5e13a2d214ed4184969df363a1aab420
 https://conda.anaconda.org/conda-forge/noarch/tqdm-4.64.1-pyhd8ed1ab_0.tar.bz2#5526ff3f88f9db87bb0924b9ce575345
 https://conda.anaconda.org/conda-forge/noarch/traittypes-0.2.1-pyh9f0ad1d_2.tar.bz2#7d32ccb5334a6822c28af3e864550618
@@ -380,7 +380,7 @@ https://conda.anaconda.org/conda-forge/noarch/aiosignal-1.2.0-pyhd8ed1ab_0.tar.b
 https://conda.anaconda.org/conda-forge/linux-64/argon2-cffi-bindings-21.2.0-py39hb9d737c_2.tar.bz2#76139de3552a2046135eb0b2d02a9c85
 https://conda.anaconda.org/conda-forge/linux-64/arrow-cpp-7.0.1-py39hd3ccb9b_1_cpu.tar.bz2#42dd96e933a6b81a27289e8575889a06
 https://conda.anaconda.org/conda-forge/noarch/async-timeout-4.0.2-pyhd8ed1ab_0.tar.bz2#25e79f9a1133556671becbd65a170c78
-https://conda.anaconda.org/conda-forge/linux-64/av-9.2.0-py39hbd8a108_2.tar.bz2#218bbfb5730e84bbdd347bdae02eaaa8
+https://conda.anaconda.org/conda-forge/linux-64/av-10.0.0-py39h169e2ed_0.tar.bz2#5612c0ac58b17c98a541615e43ecf6e6
 https://conda.anaconda.org/conda-forge/noarch/bleach-5.0.1-pyhd8ed1ab_0.tar.bz2#1f5151d37e4a2b1137f81c89a3a769f2
 https://conda.anaconda.org/conda-forge/linux-64/bottleneck-1.3.5-py39hd257fcd_0.tar.bz2#fedfa00c70e9a8c50481df1652c7fe7e
 https://conda.anaconda.org/conda-forge/linux-64/brotlipy-0.7.0-py39hb9d737c_1004.tar.bz2#05a99367d885ec9990f25e74128a8a08
@@ -498,7 +498,7 @@ https://conda.anaconda.org/conda-forge/noarch/cf_xarray-0.7.4-pyhd8ed1ab_0.tar.b
 https://conda.anaconda.org/conda-forge/noarch/descartes-1.1.0-py_4.tar.bz2#32fa3526c15250ccf353f1ce905f50b3
 https://conda.anaconda.org/conda-forge/linux-64/esmpy-8.2.0-mpi_mpich_py39h8bb458d_101.tar.bz2#347f324dd99dfb0b1479a466213b55bf
 https://conda.anaconda.org/conda-forge/linux-64/fiona-1.8.22-py39h80939cc_0.tar.bz2#2b328b9f2e4334c8b5b5ed86149460f0
-https://conda.anaconda.org/conda-forge/noarch/flox-0.6.0-pyhd8ed1ab_0.tar.bz2#7d29ed95fe16e1e21bb5af9ac49cd8bf
+https://conda.anaconda.org/conda-forge/noarch/flox-0.6.1-pyhd8ed1ab_0.tar.bz2#cfd96581514f1675056e0d1973e5495d
 https://conda.anaconda.org/conda-forge/linux-64/graphviz-6.0.1-h5abf519_0.tar.bz2#123c55da3e9ea8664f73c70e13ef08c2
 https://conda.anaconda.org/conda-forge/linux-64/grpc-google-iam-v1-0.12.4-py39hf3d152e_0.tar.bz2#5b46568e225a39302d22840ef2ed51c7
 https://conda.anaconda.org/conda-forge/noarch/ipython-8.5.0-pyh41d4057_1.tar.bz2#83600908360af344d9324c3409e0dfea
@@ -570,7 +570,7 @@ https://conda.anaconda.org/conda-forge/noarch/google-auth-httplib2-0.1.0-pyhd8ed
 https://conda.anaconda.org/conda-forge/noarch/google-auth-oauthlib-0.5.3-pyhd8ed1ab_0.tar.bz2#59c47019bc497f23971b2e4802da18b6
 https://conda.anaconda.org/conda-forge/noarch/intake-0.6.6-pyhd8ed1ab_0.tar.bz2#1dc29fb9182b0f3ce935a8bd4c341ed5
 https://conda.anaconda.org/conda-forge/noarch/ipyleaflet-0.17.1-pyhd8ed1ab_0.tar.bz2#71272260804d21e74e987831022da3ea
-https://conda.anaconda.org/conda-forge/noarch/ipyspin-0.1.2-pyh9f0ad1d_0.tar.bz2#a13e885fa5b45dc4e46ebca75615618b
+https://conda.anaconda.org/conda-forge/noarch/ipyspin-1.0.1-pyhd8ed1ab_0.tar.bz2#cb4c1f334457ef5f88458c94c503f89e
 https://conda.anaconda.org/conda-forge/noarch/ipytree-0.2.2-pyhd8ed1ab_0.tar.bz2#5cc19cec6c4598183f4a8278e2142810
 https://conda.anaconda.org/conda-forge/noarch/ipyurl-0.1.2-pyh3684270_1.tar.bz2#a72d315bc1ed6ddf0b2ace197b10886b
 https://conda.anaconda.org/conda-forge/noarch/jupyter-resource-usage-0.6.3-pyhd8ed1ab_0.tar.bz2#9d26ebd67e0e22aba13ac8a490d88d9e
@@ -581,7 +581,7 @@ https://conda.anaconda.org/conda-forge/noarch/metpy-1.3.1-pyhd8ed1ab_0.tar.bz2#d
 https://conda.anaconda.org/conda-forge/noarch/msal_extensions-1.0.0-pyhd8ed1ab_0.tar.bz2#db0e3cc6e5e99baf237a2ab73898f553
 https://conda.anaconda.org/conda-forge/noarch/msrest-0.7.1-pyhd8ed1ab_0.tar.bz2#a1733821b05a030e805e9a47e48df636
 https://conda.anaconda.org/conda-forge/noarch/nbconvert-7.2.1-pyhd8ed1ab_0.tar.bz2#8e12f153954596420bab7cdeeabb3f15
-https://conda.anaconda.org/conda-forge/noarch/notebook-shim-0.1.0-pyhd8ed1ab_0.tar.bz2#3a8e2c7dcc674f2cb0784f1faba57055
+https://conda.anaconda.org/conda-forge/noarch/notebook-shim-0.2.0-pyhd8ed1ab_0.tar.bz2#45b287252fd17eccb002676623ba859c
 https://conda.anaconda.org/conda-forge/noarch/panel-0.14.0-pyhd8ed1ab_0.tar.bz2#e0edddb6a0b42ad043511c5e2811511e
 https://conda.anaconda.org/conda-forge/linux-64/parcels-2.4.0-py39hf3d152e_0.tar.bz2#bf38ba2fb1ebbc74e1faa402dae7283c
 https://conda.anaconda.org/conda-forge/noarch/pop-tools-2021.5.28-pyhd8ed1ab_0.tar.bz2#a6abe9fd94707142081e479d97964dfb

forge/packages.txt

@@ -27,7 +27,7 @@ async-timeout==4.0.2
 async_generator==1.10
 atk-1.0==2.36.0
 attrs==22.1.0
-av==9.2.0
+av==10.0.0
 aws-c-cal==0.5.11
 aws-c-common==0.6.2
 aws-c-event-stream==0.2.7
@@ -135,7 +135,7 @@ ffmpeg==4.4.2
 findlibs==0.0.2
 fiona==1.8.22
 flit-core==3.7.1
-flox==0.6.0
+flox==0.6.1
 folium==0.13.0
 font-ttf-dejavu-sans-mono==2.37
 font-ttf-inconsolata==3.000
@@ -170,7 +170,7 @@ geotiff==1.7.1
 geoviews-core==1.9.5
 gettext==0.21.1
 gflags==2.2.2
-gh==2.17.0
+gh==2.18.0
 gh-scoped-creds==4.1
 giflib==5.2.1
 git-lfs==3.2.0
@@ -241,7 +241,7 @@ intake-stac==0.4.0
 intake-xarray==0.6.1
 ipykernel==6.16.0
 ipyleaflet==0.17.1
-ipyspin==0.1.2
+ipyspin==1.0.1
 ipython==8.5.0
 ipython_genutils==0.2.0
 ipytree==0.2.2
@@ -400,7 +400,7 @@ nettle==3.8.1
 networkx==2.8.7
 noise==1.2.2
 notebook==6.5.1
-notebook-shim==0.1.0
+notebook-shim==0.2.0
 nspr==4.32
 nss==3.78
 numba==0.56.3
@@ -573,7 +573,7 @@ tifffile==2022.10.10
 tiledb==2.11.3
 tiledb-py==0.17.5
 timezonefinder==6.0.2
-tinycss2==1.1.1
+tinycss2==1.2.1
 tk==8.6.12
 toml==0.10.2
 tomli==2.0.1