Support for Pygwalker Data Visualizations in marimo

[Haleshot]

Is your feature request related to a problem? Please describe.
When attempting to use pygwalker within marimo (a Python notebook framework), I encountered an issue where marimo was unable to display the pygwalker visualization. Specifically, I received the error message:

Unsupported mimetype: application/vnd.jupyter.widget-view+json

This prevents users from utilizing pygwalker's data visualization capabilities within marimo notebooks.

Describe the solution you'd like
I would like pygwalker to implement support for marimo by adding either a __repr_html__ or __mime__ method to the pygwalker.api.pygwalker.PygWalker class. This would allow marimo to properly render pygwalker visualizations, as described in the marimo documentation for displaying objects.

Describe alternatives you've considered
I initially tried using pygwalker with marimo following the standard instructions provided in the pygwalker repository, similar to how it's used in Jupyter notebooks. However, this approach resulted in the aforementioned error.

Additional context
This feature request originated from an attempt to integrate pygwalker with marimo, as documented in marimo issue #2486. I got suggested filing this feature request with pygwalker to implement the necessary methods for compatibility.

Implementing this feature would greatly enhance the usability of pygwalker across different Python notebook environments, particularly benefiting users of marimo who wish to use pygwalker's data visualization capabilities.

[mscolnick]

Specifically, returning the pgwalker instance at the end of the cell just calls __repr__, when no _repr_html_ is provided. It would be great to add a _repr_html_ so that it can be rendered properly in other environments that follow this spec.

[Haleshot]

CC: @longxiaofei

[longxiaofei]

I will investigate using pygwalker in marimo.

[longxiaofei]

Hi @Haleshot @mscolnick

If we were to return only a static HTML, this might be simple.

However, pygwalker by default supports using duckdb to optimize dataframe calculations, with requests initiated from HTML, data calculations performed by the Python kernel, and results returned to HTML.

This requires the communication between HTML and the Python kernel.

Initially, pygwalker's UI communication was not designed to be responsive, but instead used a traditional request-response way to complete communication.

Therefore, pygwalker hacked ipywidgets and completed similar communication in JupyterLab.

So I think I need to do the same thing in marimo.

[manzt]

Therefore, pygwalker hacked ipywidgets and completed similar communication in JupyterLab.

It might be possible/useful to base the front-end code on anywidget's APIs since marimo (and others) support bidirectional communication with anywidgets. (disclaimer - I'm the anywidget maintainer).

ps: nice talk at VIS today and congrats on best paper award!

[longxiaofei]

@manzt

Yes, I am planning to use anywidgets to replace the previous communication method.

And try it on the marimo first.

[manzt]

sweet! ping me if you run into anything.

And try it on the marimo first.

some more context on marimo's support - https://marimo.io/blog/anywidget

[Haleshot]

Yes, I am planning to use anywidgets to replace the previous communication method.

Do let me know if I can be of any help.

[longxiaofei]

pygwalker already support marimo POC.

pip install pygwalker==0.4.9.11

related pr: #647

[Haleshot]

demo code:

import pandas as pd
from pygwalker.api.marimo import walk

df = pd.read_csv("xxx.csv")

walk(df, spec="spec.json")

import pandas as pd
import pygwalker.api.marimo as pyg

df = pd.read_csv("xxx.csv")
pyg.walk(df, spec="spec.json")

On running the demo code above, I get the following error:

Caution

UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 815177: character maps to

The error seems to arise from the two import statements:

from pygwalker.api.marimo import walk
import pygwalker.api.marimo as pyg

[longxiaofei]

Hi @Haleshot , thanks for your feedback.

It seems that I did not force the encoding method to be utf-8.

Windows uses cp1252 for encoding by default.

I will fix it immediately.

[Haleshot]

Code to reproduce for reference:

# /// script
# requires-python = ">=3.11"
# dependencies = [
#     "marimo",
#     "pandas==2.2.3",
#     "pygwalker==0.4.9.11",
#     "openai==1.52.2",
#     "polars==1.12.0",
# ]
# ///

import marimo

__generated_with = "0.9.14"
app = marimo.App(width="medium")


@app.cell(hide_code=True)
def __():
    return


@app.cell
def __(df, pd, walk):
    _df = pd.read_csv("../Exploratory-Data-Analysis/assets/books_enriched.csv")

    walk(df, spec="spec.json")
    return


@app.cell
def __(df, pd, pyg):
    _df = pd.read_csv("../Exploratory-Data-Analysis/assets/books_enriched.csv")
    pyg.walk(df, spec="spec.json")
    return


@app.cell
def __():
    # import libraries
    import marimo as mo
    import pandas as pd
    from pygwalker.api.marimo import walk
    import pygwalker.api.marimo as pyg
    return mo, pd, pyg, walk


if __name__ == "__main__":
    app.run()

[longxiaofei]

@Haleshot

try it.

pip install pygwalker==0.4.9.12

[Haleshot]

@Haleshot

try it.

pip install pygwalker==0.4.9.12

This is great, it works now! Thanks a lot.

Relevant SS:

Sample code for anyone who wants to try:

# /// script
# requires-python = ">=3.11"
# dependencies = [
#     "marimo",
#     "pandas==2.2.3",
#     "pygwalker==0.4.9.12",
#     "openai==1.52.2",
#     "polars==1.12.0",
# ]
# ///

import marimo

__generated_with = "0.9.14"
app = marimo.App(width="medium")


@app.cell
def __(pd, walk):
    _df = pd.read_csv("../Exploratory-Data-Analysis/assets/books_enriched.csv")

    walk(_df, spec="spec.json")
    return


@app.cell
def __(pd, pyg):
    _df = pd.read_csv("../Exploratory-Data-Analysis/assets/books_enriched.csv")
    pyg.walk(_df, spec="spec.json")
    return


@app.cell(hide_code=True)
def __():
    # import libraries
    import marimo as mo
    import pandas as pd
    from pygwalker.api.marimo import walk
    import pygwalker.api.marimo as pyg
    return mo, pd, pyg, walk


if __name__ == "__main__":
    app.run()

From #647 (comment):

It can only make pygwalker work normally on Marimo, and currently no responsive features have been introduced.

Wanted to ask, what is meant by this? I can see pygwalker run normally as it does so in the jupyter environment. Any "responsive" features which occur in jupyter but not in marimo that we can be made aware of?

[Haleshot]

In addition to the above comment; I was wondering if adding a relevant marimo example here - https://github.com/Kanaries/pygwalker/tree/main/examples (in addition to streamlit, jupyter, etc.) would be appreciated. I can get a PR up for the same if you'd like!

[longxiaofei]

In addition to the above comment; I was wondering if adding a relevant marimo example here - https://github.com/Kanaries/pygwalker/tree/main/examples (in addition to streamlit, jupyter, etc.) would be appreciated. I can get a PR up for the same if you'd like!

Yes, thanks in advance for your PR.

[longxiaofei]

Wanted to ask, what is meant by this? I can see pygwalker run normally as it does so in the jupyter environment. Any "responsive" features which occur in jupyter but not in marimo that we can be made aware of?

Hi @Haleshot

There may be some misunderstandings.

What I mean is that marimo is a reactive notebook, but pygwalker hasn't incorporated marimo's reactive features.

[Haleshot]

What I mean is that marimo is a reactive notebook, but pygwalker hasn't incorporated marimo's reactive features.

Got it, thanks! Do let me know if I can be of any help in relation to possible inclusion of these changes.

PS: I can close the issue now.