Pyodide/marimo support

[davidar]

Mathics very nearly supports Pyodide. I put together a quick demo of this in marimo, which uses pyodide to run Python notebooks entirely in a web browser, without requiring a backend server for execution:

https://marimo.io/p/@davidar/mathics

The only major issue I had was the fact that mathics-core lists llvmlite as a mandatory dependency, which isn't currently supported by pyodide (pyodide/pyodide#621). This meant I had to manually install all the dependencies myself to avoid this one package (otherwise you get an error like this). As far as I can tell, everything except for Compile[] is still fully functional.

Otherwise I just lightly modified some of the rendering code from mathics-django to output results in a format marimo understands (click the top line of code in the notebook to expand it). I might see if I can pull the necessary bits into a separate package, as it's useful to have outside of just the django application.

Is your feature request related to a problem? Please describe.
Being able to easily run mathics by just visiting a webpage without needing to install anything makes it a lot easier to experiment with for new users.

Describe the solution you'd like
Make llvmlite an optional dependency, so that mathics can still be installed when Compile[] isn't needed/supported (as is already the case for several other builtin functions).

Describe alternatives you've considered
As I showed above, it is possible to work around this issue, but making this dependency optional would make it a lot more convenient to install mathics.

Additional context

[davidar]

It looks like mathics.core.convert.function already has a fallback for when LLVM isn't installed, however in this case it doesn't re-export CompileError, which causes this error when initialising compilation builtins:

cannot import name 'CompileError' from 'mathics.core.convert.function' (/lib/python3.12/site-packages/mathics/core/convert/function.py)
    Not able to load mathics.builtin.compilation. Check your installation.
    mathics.builtin loads from /lib/python3.12/site-packages/mathics/core/load

[davidar]

Here's a similar example that works in JupyterLite with the pyodide kernel:

https://gist.github.com/davidar/1d05165c624ee591376397ec8eb93cd1

[rocky]

First and foremost, thanks for doing this!

The work and examples look awesome!

Personally, I am in favor of making llvmlite an optional dependency.

The Mathics3 module system is stable enough that llvmlite can be put into an external module. And in my opinion, expecting llvmlite and Cython to magically provide the work that Compile[] needs to do was a cop-out. The improvement using Compile[] is nowhere near what it needs to be nor is it very effective.

Also, on the roadmap coming up whenever @mmatera has time for it is a revamp of the way that boxing and formatting work. And a big part of this is precisely so that interaction with Jupyter will be less kludgy.

@mmatera - your thoughts?

[rocky]

On further thought, the Mathics3 modules still have this issue where the scope is not quite right yet. However, I think this is still easily doable inside mathics-core by allowing llvmlite to be optional.

[mmatera]

@davidar, thanks! the example is super interesting!
Regarding llvm, #1101 should be enough to remove the dependency. Some time ago I started to decoupled it, but we still need it for a part of the implementation of Plot-like, and numerical functions. Now, it is not needed anymore.

[rocky]

@davidar, thanks! the example is super interesting! Regarding llvm, #1101 should be enough to remove the dependency. Some time ago I started to decoupled it, but we still need it for a part of the implementation of Plot-like, and numerical functions. Now, it is not needed anymore.

@mmatera Wow - that was fast!
@davidar please let us know if this does it for you - thanks.

[davidar]

Thanks for fixing this so quickly, I uploaded the wheel to https://davidar.github.io/jupyterlite-demo/lab/index.html?path=mathics.ipynb and now it can just be %pip install'd! I still see some errors when initialising the builtins, but otherwise everything seems to be working :)

/lib/python3.12/site-packages/mathics/builtin/quantum_mechanics/angular.py:89: SyntaxWarning: invalid escape sequence '\['
  """

[Errno 28] Invalid argument: '.'
    Not able to load mathics.builtin.file_operations.file_properties. Check your installation.
    mathics.builtin loads from /lib/python3.12/site-packages/mathics/core/load
[Errno 28] Invalid argument: '.'
    Not able to load mathics.builtin.exp_structure.size_and_sig. Check your installation.
    mathics.builtin loads from /lib/python3.12/site-packages/mathics/core/load
[Errno 28] Invalid argument: '.'
    Not able to load mathics.builtin.pympler.asizeof. Check your installation.
    mathics.builtin loads from /lib/python3.12/site-packages/mathics/core/load

[rocky]

As for:

...  SyntaxWarning: invalid escape sequence '\['"""

This is a known problem and something that we need to make a concerted effort to find and fix, at some point. Usually though, this happens before a release.

However,

Errno 28] Invalid argument: '.'
    Not able to load mathics.builtin.exp_structure.size_and_sig. 

is more troubling. But pympler seems like it was copied from somewhere and is very old Python code going back to 2016. I suspect it doesn't work in your environment. Could you check to see if there is a corresponding newer Pymper that works?

And asizeof is probably failing because it imports pympler.

Off hand neither of these I think is used that much. But of course, we should fix this once we know what's up.

[davidar]

Hm, I just tried running it again, and I'm no longer getting those errors, and the builtin that depends on pympler seems to be working:

So I'm not sure what was happening initially...

[davidar]

Yeah, I haven't been able to reproduce it again, everything seems to be working perfectly now, so I'm going to call this issue resolved. Thanks again for the quick turnaround, looking forward to the next release :)

Also, on the roadmap coming up whenever @mmatera has time for it is a revamp of the way that boxing and formatting work. And a big part of this is precisely so that interaction with Jupyter will be less kludgy.

That would be quite handy. Is there an issue I can follow about this?

[rocky]

#961

Yeah, I haven't been able to reproduce it again, everything seems to be working perfectly now, so I'm going to call this issue resolved. Thanks again for the quick turnaround, looking forward to the next release :)

My view right now is that we should do this before revamping boxing and formatting which may go on for a while since it is a big task, and may have to live in a branch for a while. But I would love to hear @mmatera;s thoughts.

The work that has gone on since the 7.0.0 is also major and API breaking. It might have been nice to contain the API breakage in fewer releases. But right now, I don't think this is feasible. We will also probably need to redo stuff after Python 3.13 hits which is usually near Christmas to the New Year. Again this is tentative based on what mmatera has to say.

I forked your workspace and tried it which is neat. What can be done to make it easier to package or make it easier to use Pyodide and/or marimo, short of forking workspaces?

Also, on the roadmap coming up whenever @mmatera has time for it is a revamp of the way that boxing and formatting work. And a big part of this is precisely so that interaction with Jupyter will be less kludgy.

That would be quite handy. Is there an issue I can follow about this?

#961 mentions this, and this links to FUTURE.rst which I guess I should update before 2024 ends. :-(

But again, since mmatera is driving this effort, I leave it to him to decide what to do and how to proceed on this.

[davidar]

My view right now is that we should do this before revamping boxing and formatting which may go on for a while since it is a big task, and may have to live in a branch for a while.

Ah, right. Well, the remaining pain point is getting results as SVG/HTML/TeX/etc without needing all the boilerplate I copied from mathics-django. Half of it was just for rendering graphs, so perhaps that could be pushed into pymathics-graph itself?

As for the rest of it... Ideally there would be some way for the frontend to just specify which formats it can accept (e.g. marimo uses KaTeX, which can only accept TeX format input and not MathML, unlike mathjax used by mathics-django). Then have some API which figures out the best format to convert results into based on this (i.e. without having to manually go through all the different cases)?

It'd also be useful if there was more of a separation between MathML and HTML outputs. MathML is only really usable with mathjax (Firefox is the only browser that supports it natively and even then not all that well). However, the TeX output often uses features that aren't available in katex (e.g. includegraphics, asymptote). Hence I ended up using MathML outputs for some of the examples (since that seems to be the only way to output SVG embedded inline in text at the moment?) and doing some coarse string substitutions to get things to render reasonably. It'd be nice if there was a better way of handling this.

Anyway, those are my broad thoughts, I might try playing with the code a bit more to suggest something more concrete. Any pointers to things I may have missed would be greatly appreciated, as I'm not particularly familiar with everything mathics' api provides currently.

[rocky]

My view right now is that we should do this before revamping boxing and formatting which may go on for a while since it is a big task, and may have to live in a branch for a while.

Ah, right. Well, the remaining pain point is getting results as SVG/HTML/TeX/etc without needing all the boilerplate I copied from mathics-django. Half of it was just for rendering graphs, so perhaps that could be pushed into pymathics-graph itself?

pymathics-graph is really a wrapper around networkx; it is focused on providing Mathics3 builtin functions for network or graph theory kinds of things.

But your point is well taken. That boilerplate code is also copied inside mathicsscript the CLI front end as well, with some modifications.

We probably need to isolate this aspect and put it in its own repository somewhere. However the "Mathics3 front-end API" such as it is, right now is pretty haphazard. And I am not totally sure we know what the right API is right now. We are struggling with the Boxing and Formatting aspects still. (What we do know though, is that what we have is not very good.)

As for the rest of it... Ideally, there would be some way for the frontend to just specify which formats it can accept (e.g. marimo uses KaTeX, which can only accept TeX format input and not MathML, unlike mathjax used by mathics-django). Then have some API that figures out the best format to convert results into based on this (i.e. without having to manually go through all the different cases)?

I am assuming by TeX you mean LaTeX, as opposed, to "plain TeX"? Built in, right now is TeXForm which is used primarily in formatting the Mathics3 PDF.

However, in the back of my mind, I've been thinking about the idea of adding repositories for specific kinds of forms. In particular one for converting to Python (with options to favor or include the use of SymPy, mpmath and scikit, NumPy, ... libraries).

One might do the same thing for TeX with variants for LaTeX, MathJax, KaTeX, plain, etc.

It'd also be useful if there was more of a separation between MathML and HTML outputs. MathML is only really usable with mathjax (Firefox is the only browser that supports it natively and even then not all that well). However, the TeX output often uses features that aren't available in katex (e.g. includegraphics, asymptote). Hence I ended up using MathML outputs for some of the examples (since that seems to be the only way to output SVG embedded inline in text at the moment?) and doing some coarse string substitutions to get things to render reasonably. It'd be nice if there was a better way of handling this.

Yes, it would be nice. The details need to be worked out. And we already have several large problems that are further along in the design aspect and just need work to get them finished. Sigh.

Anyway, those are my broad thoughts, I might try playing with the code a bit more to suggest something more concrete. Any pointers to things I may have missed would be greatly appreciated, as I'm not particularly familiar with everything Mathics' api provides currently.

I have linked to the PDF above, here is the developers guide

[mmatera]

@davidar, regarding creating a Pyodide package I am totally in favor of it. I think you already did most of the work to have a usable version: just put the code you pulled from mathics-django in a package. Both codes (Django and yours) could be simplified afterward using MathicsSession objects, which already implement much of the mathics_eval code.

Regarding formatting, as @rocky said, I was trying for a while to make the formatting mechanism in Mathics more similar/compatible with the one in WL. Among other things, this would allow loading specific formatting rules for each different front-end, and make more scalable and flexible the conversion into different formats. However, since this requires making some large disruptive changes along mathics-core and other packages, I still have this on my TODO list until I have the time to start and finish it.

In any case, it is something that when finished, will allow to DRY and simplify the formatting code on each front-end, but it does not avoid to use the current code to write specific formatting code. It just makes the task more hacky...

[davidar]

Thanks for the pointers, I'll have a look into it.

However, in the back of my mind, I've been thinking about the idea of adding repositories for specific kinds of forms. In particular one for converting to Python (with options to favor or include the use of SymPy, mpmath and scikit, NumPy, ... libraries).

Another one I'm interested in is outputting graphics to D3. It has a lot of nice plotting features easily available, and works quite nicely for live animations. I prototyped a thin Wolfram-style API wrapper here (though still missing a lot of features):

https://observablehq.com/@davidar/wolfram-graphics

The one issue I had (other than not wanting to reimplement all the built-in functions that mathics already has) is computing bounding boxes for Graphics scenes. D3 provides very little help for this, and my naive implementation causes things to rescale strangely during animations, so I might have to look into how mathics handles this in a bit more detail...

[davidar]

I moved all the formatting code into a module/extension that can be loaded like so:

For the moment it's living here: https://github.com/davidar/mathics-core/tree/frontend/mathics/frontend

I can split it into a separate package once I finish cleaning things up and implementing the other frontends, this was just the easiest place to put it for now.

[rocky]

The one issue I had (other than not wanting to re-implement all the built-in functions that mathics already has) is computing bounding boxes for Graphics scenes. D3 provides very little help for this, and my naive implementation causes things to rescale strangely during animations, so I might have to look into how mathics handles this in a bit more detail...

Some thoughts on this...

One reason one would (re)implement the built-in functions in the frontend (D3 here?) would be to take advantage of GPU capabilities that the user may have and that D3 and more generally nodejs/Javascript/webasm may have built in.

Right now, Python doesn't hook into GPU capabilities, as far as I know. Mathics has very little to no support for making use of multiple threads. In fact, Python has this GIL problem that perhaps will be fixed in the future.

As for computing bounding boxes for Graphics - basically, you have to compute the function to get a bounding box. In some cases one may know the bounds of one or both axes. Since our API is pretty much nonexistent, of course we can make sure to add this.

Lastly, one of the too many problems of Mathics3 is in producing graphics. Right now we have too many conversions to and from Mathics3's internal representation of literal data (the plot points for example) and Python's native representation which can get converted to JSON or whatever wire format you want much more quickly.

[rocky]

I moved all the formatting code into a module/extension that can be loaded like so:

Thanks again, for doing this!

For the moment it's living here: https://github.com/davidar/mathics-core/tree/frontend/mathics/frontend

I can split it into a separate package once I finish cleaning things up and implementing the other frontends, this was just the easiest place to put it for now.

I would be happy to create a new project in the Mathics3 space and assign you to be the onwer/maintainer.

[rocky]

@davidar One other thought on implementing graphics functions in D3 on the client versus on the Mathics3 (server) side.

First, please keep in mind that in an interactive session, there are broadly two phases: first, you enter the expression and it gets (parsed and) evaluated. The front end then gets the evaluated expression back. At this stage, the result is Box, Form, and format independant. So something like a 3D sphere just appears as Sphere[x,y,z] - the x, y, and z values get filled out with default values 0, 0, 0 out if you omitted them initially. However, no attempt to box or draw this is made.

At this point, the front end suggests what formats it can handle for the object. It might handle SVG and LaTeX but not never Javascript, D3, or matplotlib. Or, maybe it is the other way around. Or maybe it can't handle Annulus in LaTeX but for SVG it can, although it can handle Sphere for LaTeX.

And then in the boxing and formatting phase, the server code also gets to decide which of the several possible formats it has been offered to use. It does this based on its capabilities and features of the expression to be boxed and formatted.

In the phase where an expression is returned and the front end wants to box and format the expression, we only allow, a form and formatters to be indicated, e.g. TeX, SVG, text, etc. Possibly, we should allow more level of specificity by indicating for specific format the functions that are acceptable to the front end.

In this way, a front end handling D3 could note that it has Sphere implemented (or implemented super fast) in D3, but not Annulus. Or it may decide in the Animate context it wants to handle more of the built-in functions, however slow they are than it would otherwise.

This way a front end can grow its capabilities over time based on need and programmer time.

Your thoughts?

[davidar]

I would be happy to create a new project in the Mathics3 space and assign you to be the onwer/maintainer.

Cool, thanks, I'll let you know when I'm ready to package it up.

Your thoughts?

Mm, I've been thinking about the two-phase process and the relationship between graphics generation and code generation (as you mentioned forms for converting to Python code earlier). Previously I'd been thinking about graphics in the traditional sense of you write a program and hand it off to the interpreter, which executes everything to render the image and hand it back to you. But the symbolic nature of everything here means it's more like a multi-pass compiler: in the first pass everything gets expanded into primitives, constant expressions get evaluated, etc; and then the second pass lowers that intermediate representation via whatever "backend" form you've selected.

So you're not really limited to the server sending back static data for the client to simply display. It's more like, the client provides a certain computational environment, and the server then compiles the user's program into an executable that can run in that environment. In the simplest case, that's something like an SVG, but it could also be e.g. a javascript program running in a tight loop to perform all the frame-by-frame calculations required to render an animation as efficiently as possible. (After all, TeX is a programming language too.) So in that sense main job is communicating to the server all the capabilities that the client environment has, so that it knows what code generators it's able to employ for that target. The server's job isn't really to execute the program, it's just to compile it into a form that the client is able to execute.

I think I'm rambling a little unconstructively as it's late and my thoughts haven't crystallised fully yet, I'll have another go over the weekend with a clearer head ;)

In the meantime, here's another integration, Mathics in Observable: https://observablehq.com/@davidar/mathics

[davidar]

I updated my JupyterLite config to be a bit more like SymPy Live, so you can just directly enter Mathics expressions into the REPL:

• Mathics Live (takes about 30s to load)

[mmatera]

Really nice!

Plot3D does not work, but the rest looks amazing!

[davidar]

Fixed!

[davidar]

As for computing bounding boxes for Graphics - basically, you have to compute the function to get a bounding box. In some cases one may know the bounds of one or both axes. Since our API is pretty much nonexistent, of course we can make sure to add this.

I think an implementation of RegionBounds would help a lot with this. I might look into that.

First, please keep in mind that in an interactive session, there are broadly two phases: first, you enter the expression and it gets (parsed and) evaluated. The front end then gets the evaluated expression back. At this stage, the result is Box, Form, and format independant.

Picking up my train of thought from the other night: I think the fact that we have this two-stage process allows us to do things which are usually non-trivial in many programming environments. In particular, the first-stage evaluation can be completely separated from the second-stage formatting, since the only thing that needs to pass between them is the evaluated/simplified M-Expression. Conveniently this can be serialised with FullForm and deserialised on the other end with a very simple parser. It's not even really necessary to format this into an intermediate format such as JSON like the Graphics3D viewer does, we already have a wire format for free.

I threw together a prototype of this idea here: https://observablehq.com/@davidar/mathics-client-side-graphics

Here, the mathics "server" is just outputting FullForm expressions, which then gets parsed and translated to D3 in the client. The client-side code is just a relatively simple wrapper, as the mathics server is already doing all the heavy-lifting transforming the user input into a simplified and standardised form.

The cool part is that, now that all the rendering is done on the client, it's easy to do interactive animations - just send an expression with unknown variables in it, and let the client fill them in! Demo here:

https://observablehq.com/@davidar/mathics-client-side-animation

[rocky]

I think an implementation of RegionBounds would help a lot with this. I might look into that.

Thanks for thinking about this and taking an interest in our little underpowered project. The stuff you have been doing is really cool!

If you need help with RegionBounds, just ask.

I suspect that to handle everything will be a lot of work. And I can envision a special kind of "box evaluation" mode where we don't return the data points per se but have to compute the function. But if we get a few down and a pattern laid out, others will be able to fill things out.

And I think that the implementations in D3 of various functions will be useful in a mode down the line where there is mixed-mode evaluation/formatting.

[mmatera]

I think an implementation of RegionBounds would help a lot with this. I might look into that.

Great! If you need help with this, I am in to help.

First, please keep in mind that in an interactive session, there are broadly two phases: first, you enter the expression and it gets (parsed and) evaluated. The front end then gets the evaluated expression back. At this stage, the result is Box, Form, and format independant.

Picking up my train of thought from the other night: I think the fact that we have this two-stage process allows us to do things which are usually non-trivial in many programming environments. In particular, the first-stage evaluation can be completely separated from the second-stage formatting, since the only thing that needs to pass between them is the evaluated/simplified M-Expression. Conveniently this can be serialised with FullForm and deserialised on the other end with a very simple parser. It's not even really necessary to format this into an intermediate format such as JSON like the Graphics3D viewer does, we already have a wire format for free.

There is a detail with this: WMA is flexible enough to allow the user to modify how expressions are formatted after the evaluation. So, for instance, If you want to put a red box around the Integral symbol each time the result of an evaluation contains the expression Integrate[f_,x_] you can do that. Then, what the front-end receives is not a string with the FullForm representation, but a boxed expression, generated in the formatting round, by applying format rules provided both by the system, and by the user.

I threw together a prototype of this idea here: https://observablehq.com/@davidar/mathics-client-side-graphics

Good!

Here, the mathics "server" is just outputting FullForm expressions, which then gets parsed and translated to D3 in the client. The client-side code is just a relatively simple wrapper, as the mathics server is already doing all the heavy-lifting transforming the user input into a simplified and standardised form.

So, all you mention is OK, except for the format that you would send to the client.

This is how an animation looks in WMA, after formatting:

In[1]:= Animate[Plot[Sin[x-a],{x,0,2}],{a,0,2}]//ToBoxes                        

Out[1]= TagBox[StyleBox[
DynamicModuleBox[{a$$ = 0, Typeset`show$$ = True, 
       Typeset`bookmarkList$$ = {}, Typeset`bookmarkMode$$ = Menu,  
       Typeset`animator$$, Typeset`animvar$$ = 1,  
       Typeset`name$$ = "untitled", Typeset`specs$$ = {{Hold[a$$], 0, 2}}, 
       Typeset`size$$ = Automatic, Typeset`update$$ = 0, 
       Typeset`initDone$$ = False, Typeset`skipInitDone$$ = True},  
      DynamicBox[Manipulate`ManipulateBoxes[1, StandardForm, 
        Variables :> {a$$ = 0}, ControllerVariables :> {}, 
        OtherVariables :> 
         {Typeset`show$$, Typeset`bookmarkList$$, Typeset`bookmarkMode$$, 
          Typeset`animator$$, Typeset`animvar$$, Typeset`name$$, 
          Typeset`specs$$, Typeset`size$$, Typeset`update$$, 
          Typeset`initDone$$, Typeset`skipInitDone$$},
        Body :> Plot[Sin[x - a$$], {x, 0, 2}], 
        Specifications :> 
         {{a$$, 0, 2, AppearanceElements -> 
            {ProgressSlider, PlayPauseButton, FasterSlowerButtons, 
             DirectionButton}}}, 
        Options :> 
         {ControlType -> Animator, AppearanceElements -> None, 
          DefaultBaseStyle -> Animate, DefaultLabelStyle -> AnimateLabel,  
          SynchronousUpdating -> True, ShrinkingDelay -> 10.},  
        DefaultOptions :> {}], SingleEvaluation -> True], 
      DynamicModuleValues :> {}, Deinitialization :> None, 
      UntrackedVariables :> {Typeset`size$$}, 
      SynchronousInitialization -> True, 
      UnsavedVariables :> {Typeset`initDone$$}, 
      UndoTrackedVariables :> {Typeset`show$$, Typeset`bookmarkMode$$}],  
     Animate, Deployed -> True, StripOnInput -> False], 
     Manipulate`InterpretManipulate[1]]

which is similar to what you said, but in the "Box-sublenguage"

The cool part is that, now that all the rendering is done on the client, it's easy to do interactive animations - just send an expression with unknown variables in it, and let the client fill them in! Demo here:

https://observablehq.com/@davidar/mathics-client-side-animation

[davidar]

Thanks for clarifying the role of boxing here. I've lost a bit of steam on the frontend side of things, and have been poking around a bit more on working out which commonly used builtins are currently missing from mathics. I'm still a little unclear on the line between what goes in core and what's better off going in a pymathics module, what's the current thinking around that?

[rocky]

Thanks for clarifying the role of boxing here.

The roadmap for the future lists @mmatera to work on prerequisites to improve this work. So this kind of thing is best left for later anyway. If we get the simple cases of animation and graphics display working better, that would be awesome.

I've lost a bit of steam on the frontend side of things, and have been poking around a bit more on working out which commonly used builtins are currently missing from mathics.

Thanks for not giving up on this underpowered project.

I'm still a little unclear on the line between what goes in core and what's better off going in a pymathics module, what's the current thinking around that?

If what you are doing requires a dependency on a library that might need a little work to get installed, like NLP or is one alternative among several, like matplotlib for graphics rendering graphs, then it is better suited for a Mathics3 module. (I have been shying away from the term "pymathics module").

If the thing you want to add is listed as a Wolfram Language Builtin Function, then it goes in Mathics core.

Formats/Functions like ToPython[] and ToSymPy[] are not Wolfram Builtin Functions, but instead, a natural kind of extension. So that is a Mathics3 Module. The debugger stuff I am working on also doesn't follow Wolfram Language behavior, so that is also a Mathics3 Module.

I have updated the section on Mathics3 Modules to include the above.