Plot API

[rcoreilly]

This thread is to define the Plot API, for the goal / math / data system.

The classic issues apply here: imperative, code-based vs. declarative; tab completion for parameters, etc. In general the guiding principle is that closures setting struct based parameter state provide the best overall solution. The current practice often falls to string-based metadata -- won't be hard to improve on that!

The relevant standard for what people likely already know is matplotlib: https://matplotlib.org/stable/plot_types/index.html

• Imperative API that doesn't even have an explicit object that is created: plt.plot(x, y) where plt is the package import path. plt.show() generates the plot (where does it even go?)
• Massive use of all the flexible python named args, with multiple ways of doing things using shortcuts etc.
• Different functions for different plot types (plot, scatter, bar) and presumably a mix of shared and unique parameters for each plot type.

Overall, there are 2 major categories of usage:

• Code-based plotting of specific tensor data, for a specific analysis, often from an interactive REPL session where you are exploring different things, and also for final publication-quality plots with multiple sub-plots etc.
• Interactive GUI-based exploration of a big table of data with lots of data columns (variables), per the standard emergent sims. In the sims, we want to provide parameters for different variables (ranges, display params) and overall plot params, while also allowing users to select what to look at etc. This overall requires a declarative, struct of params solution.

The existing plot base api, copied from the gonum package, looks like this:

plt := plot.New()
plt.Add(plots.NewLine(xy)) // xy must implement the XYer interface - tensor by itself doesn't do it.

and if you want to parameterize things, you do it with properties on the elements:

plt := plot.New()
ln := plots.NewLine(xy)
ln.LineStyle.Width.Pt(2)
ln.LineStyle.Color = colors.Red
plt.Add(ln)

with shared Style elements like LineStyle, PointStyle, TextStyle etc used throughout.

Here's a plausible code-based approach with setter functions:

plt := plot.New()
plt.Add(plots.NewLine(x, y).Line.SetWidth(2).SetColor(colors.Red))

The tricky issue here is that the properties are organized by sub-structs, which makes it impossible to chain across multiple such elements. Could have global setters that would have to be redundantly defined to chain across sub-structs.

A styler closer could be used instead:
And a Style(func(ln *plots.Line)) would also be available for closure-based styling:

plt.Add(plots.NewLine(x, y).Style(func(ln *plots.Line) {
    ln.Line.SetWidth(2).SetColor(colors.Red)
    ln.Point.SetSize(4)
})

[kkoreilly]

How does your proposal relate to the existing struct-based plotcore.PlotEditor configuration? Are you proposing that plotcore.ColumnOptions should be replaced with plots.Line? Neither of the two APIs you proposed look particularly incredible to me; the initial chaining approach doesn't really work (as you mentioned), and adding style closures for an on-the-fly concise configuration pathway is unideal. If the only point of the Style closure is to have multiple lines, then you could just make the line object as a variable and set the parameters directly before adding it to the plot:

ln := plots.NewLine(x, y)
ln.Line.SetWidth(2).SetColor(colors.Red)
ln.Point.SetSize(4)
plt.Add(ln)

I think that proposal is better, but still much less concise than matplotlib.

[rcoreilly]

How does your proposal relate to the existing struct-based plotcore.PlotEditor configuration? Are you proposing that plotcore.ColumnOptions should be replaced with plots.Line?

Probably we want to end up with a struct of widely-applicable plot.PlotOptions that are consumed by both plots.Line and PlotEditor.

One key driver here is that for the sims logging functionality, and probably various other similar cases where some kind of hard-coded algorithm is generating data, you want to put the plot config onto the tensor of data itself (as metadata), because that is what the algorithm directly generates, and you want to put all the config stuff in one place, instead of having to go back and annotate it later by looking everything up again and adding plot config stuff somewhere else.

So having a closure mechanism is key for that functionality, replacing the horrible string-based metadata we have now.

If we have one unified set of plot.PlotOptions (or PlotStyles ?) then you can set this one metadata closure and it will properly style either a ploteditor driven by a table that includes a given tensor, or an exploratory plot you code yourself in a REPL context.

In any case, nothing would ever prevent the "manual" approach you specified.

[rcoreilly]

As for matplotlib concision, the question is whether we want to adopt some kind of additional shortcut mechanism for setting styles, using something like their goofy strings etc.

[runrc]

Some possible additional requirements,

1. Plots should support real-time updates. For example, a CPU monitor or memory usage plot that is updated possibly multiple times per second would allow building of real-time dashboards.

2. Plots should support linked charts. Let say I hover / highlight over a point on one chart, I would like the same point hovered / highlighted on another chart, based on its X-axis value.

3. Plots should support multiple Y-axis. For example, it would be useful to allow multiple Y-axis plots on a single chart all sharing the same X-axis. This is very useful for plotting multiple time series.

4. Selectable series via the legend. All charts should display a legend, however de-selecting a series from the legend would remove the plot for that series.

5. Multiple plot type. The ability to support at least the following,

• line based
• scatter plots
• bubble charts
• bar chars

6. The ability for users to create their own plot renderers would very useful. Especially for example in financial charts. The ability to plot OHLC and candlestick charts would be useful.

Please excuse me if you already support some of the above. Thank you.

[kkoreilly]

Yes, we support some of those features, and we should be able to support the rest of them soon. Thank you for that information; it is in helpful in determining what we need to work on.

[runrc]

Additionally, it would be great to see heat maps and calendar heat maps.

[kkoreilly]

Yes, the gonum plot package that our plotting was originally based on supports heat maps, and people have made OHLC and candlestick charts with it as well, so all of those should be possible to support soon.

[rcoreilly]

Here's the current plan, getting implemented -- input much appreciated! https://github.com/cogentcore/core/tree/goal/plot

[runrc]

Some considerations,

1. Some plots need to change the colour and shape of a rendered point/shape depending on certain criteria. For example,

• a bubble chart needs to vary the diameter of the 'bubble' based upon an associated value

• also, consider a financial candlestick chart which plots a bar and corresponding high and low 'wicks' using open, high, low and close prices, but varies the colour of the bar depending on whether the open price is higher or lower than the close price

2. Allowing the user to choose how and when to draw background grid markers and axis tick points would be useful. Additionally, the frequency of axis labels should be automatic, but should allow overriding for more custom control and behaviour.

3. Zooming - many times series plots allow zooming via the x-axis (time). It would be useful to see this level of zoom either via a horizontal scroll bar, or some other type of control which allows fast selection of zoom and/or panning for the entire time series.

[kkoreilly]

Yes, we are currently implementing support for dynamic color and size of points in plotting for bubble charts. We will add support for candlestick charts soon. It is definitely theoretically possible to customize the grid markers, ticks, and labels, and we can make the API easier for that soon. We do not currently support x-axis based zooming, but it definitely should be possible to implement. Thank you for the feedback!

[rcoreilly]

in the GUI you can zoom both axes and you have full control over the effective Axis range, supporting zooming if I understand correctly. In the current Style settings you have various parameters controlling tick axes.

[kkoreilly]

Yes, we support general GUI zoom in plots, but I believe that @runrc is referring to x-axis specific time series zooming (here is an example of the concept) (please correct me if I am wrong), which we currently do not support but certainly can.

As @rcoreilly said, we already have some parameters for ticks and axes, and we can add more as necessary.

[runrc]

Yes, we support general GUI zoom in plots, but I believe that @runrc is referring to x-axis specific time series zooming (here is an example of the concept) (please correct me if I am wrong), which we currently do not support but certainly can.

As @rcoreilly said, we already have some parameters for ticks and axes, and we can add more as necessary.

Yes I was indeed referring the x-axis specific time series zooming. I would expect the y-axis to scale automatically to fit the contents of the plot being showing as the user scrolls / zooms the x-axis.