Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Usage with PyScript #728

Closed
quasiben opened this issue Apr 20, 2023 · 3 comments
Closed

Usage with PyScript #728

quasiben opened this issue Apr 20, 2023 · 3 comments

Comments

@quasiben
Copy link

I was playing around with Pandas and S3 Parquet data and attempted to write a PyScript example to build a dashboard . I was blocked by s3fs not having a Python 3 wheel. This seems odd as pypi lists a pure python 3 wheel here:
https://pypi.org/project/s3fs/#files

error:

(PY1001): Unable to install package(s) 's3fs'. Reason: Can't find a pure Python 3 Wheel for package(s) 's3fs'. See: https://pyodide.org/en/stable/usage/faq.html#micropip-can-t-find-a-pure-python-wheel for more information.

<html>
<head>
<link rel="stylesheet" href="https://pyscript.net/latest/pyscript.css" />
<script defer src="https://pyscript.net/latest/pyscript.js"></script>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css">
</head>
<body>
<py-config type="toml">
    packages = ["s3fs"]
</py-config>

<py-script>
# Import libraries
import s3fs
</py-script>
</body>
</html>
@martindurant
Copy link
Member

I'm afraid you will not get s3fs working with pyscript, since it relies on aiohttp/asyncio. I assume it was the dependencies that prevented installation, which is correct behaviour.

Your best bet is to get the equivalent URLs (public, signed or as a bucket-website) and use HTTP via fsspec/filesystem_spec#1177 . Note that blocking binary downloads can only happen in a webworker, and all requests require CORS to be correctly configured on the bucket. For a pure async approach (in the main thread), fsspec/filesystem_spec#960 might be the better approach, but that code is even more super-alpha-scratch.

Yet another thing I've been working on, is rusty backends for fsspec, which insludes s3: rfsspec. This would need to be rebuilt specifically for wasm, and I know that some work has been done on the tokio async runtime for wasm, but I don't really know if it's enough to potentially make it workable.

@quasiben
Copy link
Author

Ah right! Forgot about the asyncio issue. Thanks for the thoughtful response!

@martindurant
Copy link
Member

Sidelong related: dask-distributed has a websocket protocol, which is what it would need for communication from pyscript. That could also solve the S3 issue, since we can then proxy client calls via workers. I'm pretty sure, though, that the thread and tornado stuff even in the client side is too much for pyscript. I'd be happy to brainstorm on what it might take to get this to work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@quasiben @martindurant