Fetch and http request

[jvailius]

A follow up on jupyterlite/pyodide-kernel#96

Description

I tried xeus because adding additional pkgs seems far easier then in pyodide. But I ran into issues with network requests.

Reproduce

Using fetch seems to work in principle but gave me this error:

import js
import json

json.loads(await (await js.fetch("http://localhost:8000/jupyter/lab/jupyter-lite.json")).text())

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
File /lib/python3.11/site-packages/IPython/__init__.py:44, in patch_asyncio.<locals>.asyncio_call(self, coro)
     41     task.add_done_callback(done_cb)
     43 raw_js_py_callback = pyjs.JsValue(py_callback)
---> 44 js_py_callback = raw_js_py_callback['__usafe_void_val_val__'].bind(raw_js_py_callback)
     45 js_promise = pyjs.js.Promise.new(js_py_callback)
     47 pyjs.js.globalThis.toplevel_promise = js_promise

AttributeError: has no attribute/key __usafe_void_val_val__

Then I tried httpx which seems to require openssl (I added it to my environment.yml). This made the kernel unusable with the condole error: LinkError: WebAssembly.instantiate(): Import #92 module="env" function="time": imported function does not match the expected type. Complete output below.

I can confirm that requests lib works but I could not test it with https so far.

Context

• JupyterLite version: 0.2.3
• Operating System and version: debian bookworm
• Browser and version: Chrome, Version 122.0.6261.111 (Official Build) (64-bit)

Browser Output

870.1260b274e8c70c25…0b274e8c70c25c3c4:2 Uncaught (in promise) LinkError: WebAssembly.instantiate(): Import #92 module="env" function="time": imported function does not match the expected type

  | deserialize | @ | 870.1260b274e8c70c25…0b274e8c70c25c3c4:2
-- | -- | -- | --
  | S | @ | 870.1260b274e8c70c25…0b274e8c70c25c3c4:2
  | Promise.then (async) |   |  
  | apply | @ | 870.1260b274e8c70c25…0b274e8c70c25c3c4:2
  | h | @ | 413.df2804f5ae3431aa…804f5ae3431aa4c17:1
  | create | @ | 413.df2804f5ae3431aa…804f5ae3431aa4c17:1
  | startNew | @ | kernels.js:123
  | startNew | @ | sessions.js:105
  | (anonymous) | @ | index.js:366
  | route | @ | router.js:85
  | await in route (async) |   |  
  | fetch | @
ia-expanded="true" tabindex="-1" style="box-sizing: border-box; min-width: 0px; min-height: 0px; flex: 0 0 auto;">
870.1260b274e8c70c25…0b274e8c70c25c3c4:2 Uncaught (in promise) LinkError: WebAssembly.instantiate(): Import #92 module="env" function="time": imported function does not match the expected type


deserialize
@
870.1260b274e8c70c25…0b274e8c70c25c3c4:2
S
@
870.1260b274e8c70c25…0b274e8c70c25c3c4:2
Promise.then (async)
apply
@
870.1260b274e8c70c25…0b274e8c70c25c3c4:2
h
@
413.df2804f5ae3431aa…804f5ae3431aa4c17:1
create
@
413.df2804f5ae3431aa…804f5ae3431aa4c17:1
startNew
@
kernels.js:123
startNew
@
sessions.js:105
(anonymous)
@
index.js:366
route
@
router.js:85
await in route (async)
fetch
@

[DerThorsten]

Hi, thanks for reporting this.

A few questions:

• Are you running this piece of code in an async function or directly in a cell?
• Also, can you check if there is some error in the console?
• Is the deployment also on localhost? If the deployment of jupyterlite is not on local host this cannot work.

A suggestion would be to just use the requests package which should just work

[jvailius]

• Are you running this piece of code in an async function or directly in a cell?

directly

• Also, can you check if there is some error in the console?

I'll check - but it should be very easy to reproduce

• Is the deployment also on localhost? If the deployment of jupyterlite is not on local host this cannot work.

localhost: same url as I am using in the request

A suggestion would be to just use the requests package which should just work

OK :)

[jvailius]

It seems there are "two" errors - at least error messages but the result is OK.

Even with requests I get the error mentioned above in the cell output if I wrap requests in an async function and await it:

AttributeError Traceback (most recent call last)
File /lib/python3.11/site-packages/IPython/init.py:44, in patch_asyncio..asyncio_call(self, coro)
41 task.add_done_callback(done_cb)
43 raw_js_py_callback = pyjs.JsValue(py_callback)
---> 44 js_py_callback = raw_js_py_callback['usafe_void_val_val'].bind(raw_js_py_callback)
45 js_promise = pyjs.js.Promise.new(js_py_callback)
47 pyjs.js.globalThis.toplevel_promise = js_promise

AttributeError: has no attribute/key usafe_void_val_val

Without async I just get an error in the browser console:

Refused to set unsafe header "Accept-encoding"
Refused to set unsafe header "Connection"

I am using async function sin a library because I use micropip and I'd like to make my library work with both kernels.

[DerThorsten]

I can confirm that toplevel awaits are not working atm
So as a temporary workaround, just wrap your cell in an async function.

Ie:

async def foo():
    # your code here
asyncio.create_task(foo())

[DerThorsten]

I will probably be able to add a fix in the next days

[jvailius]

Thank you @DerThorsten. OT: I have tested xeus vs pyodide and I hope I can keep xeus. Being able to just provide the environment without extra installs is very useful.

[DerThorsten]

this is fixed (a patch in IPython needed to be updated)

To use this, you will need the most recent build of IPython (ipython=8.22.2=py311had7285e_1).
(FYI, the most recent IPython also needs a very recent traitlets package )

This is the script I used to create working build

WASM_ENV_NAME=my-env-name
WASM_ENV_PREFIX=$MAMBA_ROOT_PREFIX/envs/$WASM_ENV_NAME
micromamba create -n $WASM_ENV_NAME \
            --platform=emscripten-wasm32 \
            -c https://repo.mamba.pm/emscripten-forge \
            -c https://repo.mamba.pm/conda-forge \
            --yes \
            xeus-python "ipython=8.22.2=py311had7285e_1" "traitlets>=5.14.2"

jupyter lite build \
        --XeusAddon.prefix=$WASM_ENV_PREFIX 

[jvailius]

Thanks for the additional hints. I'll give it a try.

[DerThorsten]

there is still a tiny issue, atm you need to import pyjs in the async cell.
This will be fixed once emscripten-forge/recipes#829 lands

[DerThorsten]

The ipython build with the fix : ipython-8.22.2-py311had7285e_2

therefore the updated snipped is:

WASM_ENV_NAME=my-env-name
WASM_ENV_PREFIX=$MAMBA_ROOT_PREFIX/envs/$WASM_ENV_NAME
micromamba create -n $WASM_ENV_NAME \
            --platform=emscripten-wasm32 \
            -c https://repo.mamba.pm/emscripten-forge \
            -c https://repo.mamba.pm/conda-forge \
            --yes \
            xeus-python "ipython=8.22.2=py311had7285e_2" "traitlets>=5.14.2"

jupyter lite build \
        --XeusAddon.prefix=$WASM_ENV_PREFIX