Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docker Image with JupySQL already set up. #996

Open
EdanSaltz opened this issue Mar 7, 2024 · 8 comments
Open

Docker Image with JupySQL already set up. #996

EdanSaltz opened this issue Mar 7, 2024 · 8 comments

Comments

@EdanSaltz
Copy link

I've tried to create a Docker container using the pyspark-notebook as a base, but am getting errors when I include 'RUN pip install jupysql.' The error is :\jupyter_server.extension.utils.ExtensionLoadingError: _load_jupyter_server_extension function was not found:

jupyter_server.extension.utils.ExtensionLoadingError: _load_jupyter_server_extension function was not found.
@edublancas
Copy link

can you share the full traceback?

@EdanSaltz
Copy link
Author

2024-03-07T14:56:25.743737504Z Entered start.sh with args: jupyter lab
2024-03-07T14:56:25.745030170Z Running hooks in: /usr/local/bin/start-notebook.d as uid: 1000 gid: 100
2024-03-07T14:56:25.745128670Z Done running hooks in: /usr/local/bin/start-notebook.d
2024-03-07T14:56:25.748737129Z Running hooks in: /usr/local/bin/before-notebook.d as uid: 1000 gid: 100
2024-03-07T14:56:25.749016712Z Sourcing shell script: /usr/local/bin/before-notebook.d/spark-config.sh
2024-03-07T14:56:25.749023462Z Done running hooks in: /usr/local/bin/before-notebook.d
2024-03-07T14:56:25.749025004Z Executing the command: jupyter lab
2024-03-07T14:56:26.353632796Z [I 2024-03-07 14:56:26.353 ServerApp]�(B Package jupyterlab took 0.0000s to import
2024-03-07T14:56:26.354059671Z [I 2024-03-07 14:56:26.353 ServerApp]�(B Package jupysql_plugin took 0.0002s to import
2024-03-07T14:56:26.359415462Z [I 2024-03-07 14:56:26.359 ServerApp]�(B Package jupyter_lsp took 0.0052s to import
2024-03-07T14:56:26.359424712Z [W 2024-03-07 14:56:26.359 ServerApp]�(B A _jupyter_server_extension_points function was not found in jupyter_lsp. Instead, a _jupyter_server_extension_paths function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
2024-03-07T14:56:26.360135879Z [I 2024-03-07 14:56:26.360 ServerApp]�(B Package jupyter_server_mathjax took 0.0006s to import
2024-03-07T14:56:26.362633671Z [I 2024-03-07 14:56:26.362 ServerApp]�(B Package jupyter_server_terminals took 0.0023s to import
2024-03-07T14:56:26.376265004Z [I 2024-03-07 14:56:26.376 ServerApp]�(B Package jupyterlab_git took 0.0134s to import
2024-03-07T14:56:26.377520629Z [I 2024-03-07 14:56:26.377 ServerApp]�(B Package nbclassic took 0.0011s to import
2024-03-07T14:56:26.378316671Z [W 2024-03-07 14:56:26.378 ServerApp]�(B A _jupyter_server_extension_points function was not found in nbclassic. Instead, a _jupyter_server_extension_paths function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
2024-03-07T14:56:26.378598129Z [I 2024-03-07 14:56:26.378 ServerApp]�(B Package nbdime took 0.0000s to import
2024-03-07T14:56:26.378727421Z [I 2024-03-07 14:56:26.378 ServerApp]�(B Package notebook took 0.0000s to import
2024-03-07T14:56:26.379926212Z [I 2024-03-07 14:56:26.379 ServerApp]�(B Package notebook_shim took 0.0000s to import
2024-03-07T14:56:26.379934712Z [W 2024-03-07 14:56:26.379 ServerApp]�(B A _jupyter_server_extension_points function was not found in notebook_shim. Instead, a _jupyter_server_extension_paths function was found and will be used for now. This function name will be deprecated in future releases of Jupyter Server.
2024-03-07T14:56:26.384199087Z [I 2024-03-07 14:56:26.384 ServerApp]�(B Package ploomber_extension took 0.0040s to import
2024-03-07T14:56:26.384295587Z [I 2024-03-07 14:56:26.384 ServerApp]�(B jupysql_plugin | extension was successfully linked.
2024-03-07T14:56:26.384320837Z [I 2024-03-07 14:56:26.384 ServerApp]�(B jupyter_lsp | extension was successfully linked.
2024-03-07T14:56:26.386109671Z [I 2024-03-07 14:56:26.385 ServerApp]�(B jupyter_server_mathjax | extension was successfully linked.
2024-03-07T14:56:26.387677837Z [I 2024-03-07 14:56:26.387 ServerApp]�(B jupyter_server_terminals | extension was successfully linked.
2024-03-07T14:56:26.390109712Z [I 2024-03-07 14:56:26.389 ServerApp]�(B jupyterlab | extension was successfully linked.
2024-03-07T14:56:26.390116171Z [I 2024-03-07 14:56:26.390 ServerApp]�(B jupyterlab_git | extension was successfully linked.
2024-03-07T14:56:26.391862796Z [I 2024-03-07 14:56:26.391 ServerApp]�(B nbclassic | extension was successfully linked.
2024-03-07T14:56:26.391869129Z [I 2024-03-07 14:56:26.391 ServerApp]�(B nbdime | extension was successfully linked.
2024-03-07T14:56:26.394393254Z [I 2024-03-07 14:56:26.394 ServerApp]�(B notebook | extension was successfully linked.
2024-03-07T14:56:26.394895421Z [I 2024-03-07 14:56:26.394 ServerApp]�(B Writing Jupyter server cookie secret to /home/jovyan/.local/share/jupyter/runtime/jupyter_cookie_secret
2024-03-07T14:56:26.516134379Z [I 2024-03-07 14:56:26.515 ServerApp]�(B notebook_shim | extension was successfully linked.
2024-03-07T14:56:26.516156671Z [I 2024-03-07 14:56:26.515 ServerApp]�(B ploomber_extension | extension was successfully linked.
2024-03-07T14:56:26.523903712Z [I 2024-03-07 14:56:26.523 ServerApp]�(B notebook_shim | extension was successfully loaded.
2024-03-07T14:56:26.524317212Z [W 2024-03-07 14:56:26.523 ServerApp]�(B jupysql_plugin | extension failed loading with message: ExtensionLoadingError('load_jupyter_server_extension function was not found.')
2024-03-07T14:56:26.524389212Z Traceback (most recent call last):
2024-03-07T14:56:26.524391921Z File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 356, in load_extension
2024-03-07T14:56:26.524393379Z extension.load_all_points(self.serverapp)
2024-03-07T14:56:26.524394379Z File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 228, in load_all_points
2024-03-07T14:56:26.524395421Z return [self.load_point(point_name, serverapp) for point_name in self.extension_points]
2024-03-07T14:56:26.524396379Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-07T14:56:26.524397212Z File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 228, in
2024-03-07T14:56:26.524398296Z return [self.load_point(point_name, serverapp) for point_name in self.extension_points]
2024-03-07T14:56:26.524399212Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-03-07T14:56:26.524406546Z File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 219, in load_point
2024-03-07T14:56:26.524407629Z return point.load(serverapp)
2024-03-07T14:56:26.524408421Z ^^^^^^^^^^^^^^^^^^^^^
2024-03-07T14:56:26.524409254Z File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 146, in load
2024-03-07T14:56:26.524410212Z loader = self.get_loader()
2024-03-07T14:56:26.524411046Z ^^^^^^^^^^^^^^^^^^
2024-03-07T14:56:26.524411879Z File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/manager.py", line 115, in get_loader
2024-03-07T14:56:26.524412837Z loader = get_loader(loc)
2024-03-07T14:56:26.524413629Z ^^^^^^^^^^^^^^^
2024-03-07T14:56:26.524414504Z File "/opt/conda/lib/python3.11/site-packages/jupyter_server/extension/utils.py", line 47, in get_loader
2024-03-07T14:56:26.524415462Z raise ExtensionLoadingError(msg) from None
2024-03-07T14:56:26.524416379Z jupyter_server.extension.utils.ExtensionLoadingError: load_jupyter_server_extension function was not found.
2024-03-07T14:56:26.525408754Z [I 2024-03-07 14:56:26.525 ServerApp]�(B jupyter_lsp | extension was successfully loaded.
2024-03-07T14:56:26.525658546Z [I 2024-03-07 14:56:26.525 ServerApp]�(B jupyter_server_mathjax | extension was successfully loaded.
2024-03-07T14:56:26.526167004Z [I 2024-03-07 14:56:26.526 ServerApp]�(B jupyter_server_terminals | extension was successfully loaded.
2024-03-07T14:56:26.527294337Z [I 2024-03-07 14:56:26.527 LabApp]�(B JupyterLab extension loaded from /opt/conda/lib/python3.11/site-packages/jupyterlab
2024-03-07T14:56:26.527300171Z [I 2024-03-07 14:56:26.527 LabApp]�(B JupyterLab application directory is /opt/conda/share/jupyter/lab
2024-03-07T14:56:26.527540754Z [I 2024-03-07 14:56:26.527 LabApp]�(B Extension Manager is 'pypi'.
2024-03-07T14:56:26.528661129Z [I 2024-03-07 14:56:26.528 ServerApp]�(B jupyterlab | extension was successfully loaded.
2024-03-07T14:56:26.530251796Z [I 2024-03-07 14:56:26.530 ServerApp]�(B jupyterlab_git | extension was successfully loaded.
2024-03-07T14:56:26.531528921Z
2024-03-07T14:56:26.531539837Z _ _ _ _
2024-03-07T14:56:26.531541171Z | | | | __ | | | | ___
2024-03-07T14:56:26.531545796Z | || | ' / / _ | / -)
2024-03-07T14:56:26.531546712Z _/| ./_,_,|____|
2024-03-07T14:56:26.531547712Z ||
2024-03-07T14:56:26.531548546Z
2024-03-07T14:56:26.531549504Z Read the migration plan to Notebook 7 to learn about the new features and the actions to take if you are using extensions.
2024-03-07T14:56:26.531555629Z
2024-03-07T14:56:26.531556504Z https://jupyter-notebook.readthedocs.io/en/latest/migrate_to_notebook7.html
2024-03-07T14:56:26.531557421Z
2024-03-07T14:56:26.531558212Z Please note that updating to Notebook 7 might break some of your extensions.
2024-03-07T14:56:26.531559129Z
2024-03-07T14:56:26.531786962Z [I 2024-03-07 14:56:26.531 ServerApp]�(B nbclassic | extension was successfully loaded.
2024-03-07T14:56:26.555726587Z [I 2024-03-07 14:56:26.555 ServerApp]�(B nbdime | extension was successfully loaded.
2024-03-07T14:56:26.557134087Z [I 2024-03-07 14:56:26.556 ServerApp]�(B notebook | extension was successfully loaded.
2024-03-07T14:56:26.557141754Z Base URL: /
2024-03-07T14:56:26.557236462Z [I 2024-03-07 14:56:26.557 ServerApp]�(B Registered ploomber-extension server extension
2024-03-07T14:56:26.557241546Z [I 2024-03-07 14:56:26.557 ServerApp]�(B ploomber_extension | extension was successfully loaded.
2024-03-07T14:56:26.557478504Z [I 2024-03-07 14:56:26.557 ServerApp]�(B Serving notebooks from local directory: /home/jovyan
2024-03-07T14:56:26.557482546Z [I 2024-03-07 14:56:26.557 ServerApp]�(B Jupyter Server 2.8.0 is running at:
2024-03-07T14:56:26.557483796Z [I 2024-03-07 14:56:26.557 ServerApp]�(B http://2442089ed2f8:8888/lab?token=8ede59748caf5e7ed37cc0d585ceecc90f2f29fa0b886d69
2024-03-07T14:56:26.557484962Z [I 2024-03-07 14:56:26.557 ServerApp]�(B http://127.0.0.1:8888/lab?token=8ede59748caf5e7ed37cc0d585ceecc90f2f29fa0b886d69
2024-03-07T14:56:26.557508921Z [I 2024-03-07 14:56:26.557 ServerApp]�(B Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
2024-03-07T14:56:26.559278587Z [C 2024-03-07 14:56:26.559 ServerApp]�(B
2024-03-07T14:56:26.559298712Z
2024-03-07T14:56:26.559300254Z To access the server, open this file in a browser:
2024-03-07T14:56:26.559301254Z file:///home/jovyan/.local/share/jupyter/runtime/jpserver-7-open.html
2024-03-07T14:56:26.559302171Z Or copy and paste one of these URLs:
2024-03-07T14:56:26.559303046Z http://2442089ed2f8:8888/lab?token=8ede59748caf5e7ed37cc0d585ceecc90f2f29fa0b886d69
2024-03-07T14:56:26.559304379Z http://127.0.0.1:8888/lab?token=8ede59748caf5e7ed37cc0d585ceecc90f2f29fa0b886d69
2024-03-07T14:56:26.777297796Z [I 2024-03-07 14:56:26.777 ServerApp]�(B Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, python-lsp-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server

@edublancas
Copy link

i think the problem might be the jupyterlab version. are you installing jupyterlab3?

if so, you need to install another jupysql-plugin version:

pip install 'jupysql-plugin<0.4'
pip install jupysql

@EdanSaltz
Copy link
Author

It didn't work. The following is my Dockerfile

FROM jupyter/pyspark-notebook:latest

COPY ./requirements.txt /tmp/requirements.txt
#RUN pip install -r /tmp/requirements.txt

RUN pip install duckdb
RUN pip install 'jupysql-plugin<0.4'
RUN pip install jupysql --upgrade
RUN pip install duckdb-engine
RUN pip install grpcio

@edublancas
Copy link

I took a deeper look. You're seeing a warning, not an error. I don't fully understand the details but it seems like an API change in Jupyter's internals (it's complaining that jupysql-plugin is missing a hook for a server extension, but jupysql-plugin is a frontend extension, not a server extension, so the whole thing is a bit confusing)

you can remove the line RUN pip install 'jupysql-plugin<0.4' because the FROM jupyter/pyspark-notebook:latest already ships JupyterLab 4.

I built the dockerfile and the plugin is working fine:

2024-03-08 7 19 17 p m

@flaviomartins
Copy link

@EdanSaltz You can take a look at my Dockerfile. Following jupyter/pyspark-notebook use mamba to install dependencies https://github.com/bdist/notebook/blob/main/notebook/Dockerfile

@EdanSaltz
Copy link
Author

I'll need to try this docker image. Seems a bit like throwing in the kitchen sink with all the different versions of software.

@EdanSaltz
Copy link
Author

@flaviomartins I was not able to get this to work. I'll try again.

@neelasha23 neelasha23 mentioned this issue May 8, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@flaviomartins @edublancas @EdanSaltz