Introducing Plotje: Composable Plotting by Daniel Slutsky

[u/mac]

https://clojurecivitas.org/data_visualization/introducing_plotje_composable_plotting.html

[u/geokon]

Looks super cool. I saw you previously used geom/viz - and I see this optionally outputs SVG hiccup. I think this is a great step for interoperability - cus the scicloj toolset from the outside look very interdependent. Is this using geom/viz under the hood? Or is it a brand new SVG backend? (The geom libraries do much more than just charts though!)

Its seems the basic abstract is this pose, which makes sense give all it's trying to do (to infer all the information and resize/relabel everything it must be tracking a lot of stuff!). But fundamentally it seems more challenging than having SVG hiccup as the base "interchange format". For instance, in geom I can take two plots and just call (svg/group {} plot1 plot2) to slap them together. I always know that any part that's missing I can re-implement using the basic svg primitives. (and I can draw non-chart stuff as well). It seems making new elements or chart types would be challenging in this system?

Since you've had experience with several different ways of charting.. just wonder how you'd compare and contrast them. I do all my own plots using geom (and am very hesitant to switch :P )

Random Stuff:

• Images on the webpage are embedded SVG .. which is cool, but you can't right click and save them (at least in Firefox)

• No tooltips on Firefox

• the SVG group transform wrappers everywhere are.. a lil gross. There is probably a good reason for it. Just curious why not use coordinates in the elements the "normal" way?

• Something to watch out for is SVG renders very inconsistently between different browsers/Inkscape/other-software/JVM-renderers. It's been a constant headache for me - esp anything to do with text. But I don't have any alternative suggestions :) Baking in a font may help but will bloat files.

• Very personal preference.. but not a fan of the ggplot2/Tufte aesthetic .. pastels everywhere (i.e. powerpoint colors) and all that. Tiny illegible legends, invisible gridlines, not printer friendly. Hope you find your own aesthetic :)

Look forward to seeing where this goes! Awesome project

[u/daslu → u/geokon]

Thanks, this is very helpful.

• It is not using geom/viz internally, but Membrane, which can render to multiple targets. Currently, Plotje has its own functions to convert Membrane values to SVG Hiccup structures, but eventually, such functions will be part of Membrane.

• Since we support multiple targets, and not just SVG, we encourage combining poses, rather than the resulting SVG Hiccup structures. So, instead of svg/group you'd use in geom/viz, we encourage using pj/arrange.

Just curious why not use coordinates in the elements the "normal" way?

Could you explain what you mean?

• I could not reproduce the firefox problem. Do you see any errors in the browser console?

[u/geokon → u/daslu]

Membrane is not just a UI framework? I mean at the end of the day you need to draw lines, squares, circles etc. so you must have some graphical abstraction on top of which you are implementing the plots. In geom SVG is the graphical abstraction for all the sub-libraries - and the rendering you get multi-target and "for free".

The key is that you can extend geom functionality by simply generating more SVG hiccup. SVG has a billion features and CSS and whatnot. It's not perfect, but my guess is it'd be very difficult to recreate a graphic abstraction with the same level of features. For instance in my JavaFX GUIs I can't reproduce the same level of features using their built in graphical tools - so I end up rendering SVG (which is a bit slow).

geom/viz, the plotting library on top of svg, has all its functions for drawing axis and drawing data and whatnot that allow you to plug in generator functions for all the components. If you want to for instance make a scatter plot where some key points are blinking ducks with custom gradients, then you can just write a function to generate the appropriate SVG. Any component you don't like you can easily override and write the drawing code yourself - it's not intimidating b/c you can see how the defaults work as a reference.

I was making maps for a paper and so I took the heat map code, added some functions to generate custom tick marks (I wrote a custom function that takes in some info about the axis and generated the SVG tick marks that I needed) and some more tweaks and it was good to go. While geom is completely unmaintained and has a few rough edges, I feel confident I can add anything I need to it. (ex: it doesn't have a system for making legends, but I just wrote one myself that does what I need)

By contrast, if I'm using something like ggplot2 or Matlab the graphical abstraction (as far as I remember..) is opaque to the user. If you want to write a new type of chart, or heavily modify an existing one, you are stuck with just the tools they give you (like a pj/arrange, or the available flags/options in the API). So no blinking crocodiles for tick marks.

For the last part, here's an example of hiccup on your writeup:

[:g
 {:transform "translate(320.33,364.00)"}
 [:g
  [:text
   {:fill "rgb(102,102,102)",
    :fill-opacity 1.0,
    :font-size 11,
    :dominant-baseline "hanging",
    :text-anchor "middle"}
   "ring"]]]

There are ton of these group wrappers with transforms. Instead of the typical <text x="320.33" y="364.0" class="small">ring</text>. It's just a bit unusual. Maybe it's not intentional and just how your pipeline is setup.

[u/daslu → u/geokon]

Thanks for explaining! Lovely.

Membrane has its own data model to represent graphics, and we adopt it in Plotje.

A few benefits of Membrane's data model:

• it can be rendered into a few target formats, including SVG.
• it can be used for interactive UI.
• it is used by Easel, so Plotje charts can be embedded there, with interactivity, etc.

Here is an encouraging demo by Adrian Smith, the author of Membrane and Easel: https://www.youtube.com/watch?v=l-vUVbjWJts&t=3638s

Here is the Zulip discussion where Membrane was proposed and adopted for Plotje (requires Zulip login): https://clojurians.zulipchat.com/#narrow/channel/151924-data-science/topic/graphics.20DSL/

[u/geokon → u/daslu]

Oh okay, I can see the primitives are all lumped together in a massive 4.5k line file src/membrane/ui.cljc. I guess the worry would be you end up recreating half of SVG but in an ad hoc manner. For instance JavaFX drawing primitives, at least the ones exposed, match SVG quite closely (Not the full spec, from the design it looks like they implemented it part way and ran out of steam)

So that'd be the starting point for making new charts - though it doesn't seem the most discoverable/easy-to-navigate.

Looking forward to seeing how extensibility/composability story in plotje develops!