Lets-Plot library plots are not rendering

[OSuwaidi]

I'm quite new to Panel and I would really like to start using it since it supports a wide range of Python plotting libraries and is excellent for dashboarding even for professional settings.

However, I happen to be predominantly using the Lets-Plot Python visualization library, which unfortunately, doesn't seem to be supported by Panel!

When I bind a panel widget to a plotting function that returns a lets-plot plot object, the plot is not rendered (laid out) upon using a Panel layout such as Column.

Here's a MRE:

from lets_plot import *
import panel as pn
import pandas as pd

LetsPlot.setup_html()
pn.extension()
df = pd.DataFrame({
    'x': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
    'y': [2, 4, 1, 8, 5, 7, 6, 3, 9, 10],
})


def create_lets_plot(rng):
    p = (ggplot(df[:rng]) + geom_line(aes(x='x', y='y')))
    return p


slider = pn.widgets.IntSlider(name="range", value=5, start=0, end=len(df))
bound_plot = pn.bind(create_lets_plot, slider)
pn.Column(slider, bound_plot)

The only way to actually visualize anything is by forcing a plot via p.show(), but that will stack the plots multiple times which is obviously undesirable.

[MarcSkovMadsen]

Hi @OSuwaidi

The plot probably need an export to SVG or HTML to be displayed in Panel.

I'll take a look when I get back to my computer. My starting point would be see how it exports it self in https://github.com/JetBrains/lets-plot/blob/master/python-package/lets_plot/export/ggsave_.py

Maybe we should add a lets-plot pane to automatically do this for you and other users?

[OSuwaidi]

Hey @MarcSkovMadsen,

Thanks for the hint, after messing around with the exports a bit, a temporary solution that seems to work is by passing the lets-plot plot object into the export_svg() function and then returning it from create_lets_plot():

def export_svg(plot: Union[PlotSpec, SupPlotsSpec, GGBunch]):
    """
    Export plot or `bunch` to a file in SVG format.

    Parameters
    ----------
    plot: `PlotSpec`, `SupPlotsSpec` or `GGBunch` object
            Plot specification to export.

    Returns
    -------
        An SVG file.

    """
    if not (isinstance(plot, PlotSpec) or isinstance(plot, SupPlotsSpec) or isinstance(plot, GGBunch)):
        raise ValueError("PlotSpec, SupPlotsSpec or GGBunch expected but was: {}".format(type(plot)))

    from lets_plot import _kbridge as kbr

    return kbr._generate_svg(plot.as_dict())

However, the generated plot(s) are static (no tooltips), and I was unsuccessful in returning the plots in an HTML format (just displays blank upon serving the app).

Appreciate if we can support the plots in an interactive/dynamic way!

[MarcSkovMadsen]

Hi @OSuwaidi

You can try using the below. If you experience issues please report back with minimum, reproducible examples

lets-plot.mp4

import param
from panel.reactive import ReactiveHTML

from lets_plot import __version__ as _lets_plot_version
from lets_plot.plot.core import PlotSpec


class LetsPlotPane(ReactiveHTML):
    object: PlotSpec = param.ClassSelector(class_=PlotSpec, precedence=-1)

    _plot_spec_as_dict = param.Dict()

    _template = '<div id="pn-container" style="height:100%;width:100%"></div>'

    __javascript__ = [
        f"https://cdn.jsdelivr.net/gh/JetBrains/lets-plot@v{_lets_plot_version}/js-package/distr/lets-plot.min.js"
    ]

    @param.depends("object", watch=True, on_init=True)
    def _update_config(self):
        if not self.object:
            self._plot_spec_as_dict = {}
        else:
            spec = self.object.as_dict()
            if "data" in spec and isinstance(spec["data"], pd.DataFrame):
                spec["data"] = spec["data"].to_dict(orient="list")

            self._plot_spec_as_dict = spec

    _scripts = {
        "render": "state.height=-10",
        "after_layout": "self._plot_spec_as_dict()",
        "_plot_spec_as_dict": """
var height=pn_container.clientHeight
if (state.height-5<=height & height<=state.height+5){height=state.height}
state.height=height
pn_container.innerHTML=""
LetsPlot.buildPlotFromProcessedSpecs(data._plot_spec_as_dict, pn_container.clientWidth-5, height, pn_container);
""",
    }


from lets_plot import ggplot, geom_line, aes
import pandas as pd
import panel as pn

pn.extension(design="bootstrap")

df = pd.DataFrame(
    {
        "x": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
        "y": [2, 4, 1, 8, 5, 7, 6, 3, 9, 10],
    }
)


def create_lets_plot(rng):
    p = ggplot(df[:rng]) + geom_line(aes(x="x", y="y"))
    return p


def create_lets_plot_pane(rng):
    p = create_lets_plot(rng)
    return LetsPlotPane(object=p, sizing_mode="stretch_width")


slider = pn.widgets.IntSlider(name="Range", value=5, start=0, end=len(df))
bound_plot = pn.bind(create_lets_plot_pane, slider)
component = pn.Column(
    pn.Row(
        "# Lets-Plot",
        pn.layout.HSpacer(),
        pn.pane.Image(
            "https://panel.holoviz.org/_static/logo_horizontal_light_theme.png",
            height=50,
        ),
    ),
    pn.layout.Divider(),
    slider,
    bound_plot,
    sizing_mode="stretch_width",
).servable()

panel serve app.py

Notes before taking this to next level

• Lets-plot does not support responsive mode. You have to specify the exact width and height in the LetsPlot.buildPlotFromProcessedSpecs.
  • view.height, view.width are null and cannot be used to set the height and width.
  • after_layout is rerun when ever the window size changes
    • after_layout is rerun in infinity if I don't use the state.height hack.
  • Lets-plot does not provide any method to reset, clear or update a plot. Thus I use .innerHTML="".
• I don't know how the LetsPlotPane could take a bound function returning a PlotSpec instead of a PlotSpec. See ReactiveHTML: Enable me to provide bound functions to parameters #5476
• If I don't set precedence=-1 on the object parameter I will get a serialization error. We should document in the ReactiveHTML docs how to best solve this issue.

[MarcSkovMadsen]

Looking at https://github.com/randyzwitch/streamlit-letsplot/blob/master/streamlit_letsplot/__init__.py i can see that both PlotSpec and GGBunch needs to be supported.

[MarcSkovMadsen]

Would Panel be interested in add a LetsPlot Pane natively @philippjfr ? Or should it be a separate extension?

[OSuwaidi]

Any updates on this @philippjfr ?

[OSuwaidi]

Hey @MarcSkovMadsen 👋🏼, upon experimenting a bit with the work around you provided, it seems to break down when using plots having their stat parameter not defaulted to 'identity'. If you try to plot geom_density() or geom_smooth() for example, it spits out an error because lets-plot starts looking for the dataframe's meta data such as ..quantile.. and ..ymin/max.. for geom_density() and geom_smooth(), respectively.

But even upon manually providing those variables by augmenting the dataframe before feeding it into ggplot(), the rendered plots just reduce to something that you would get from geom_line().

Also, for any of the ordinary plots with their stat parameter defaulted to 'identity', if you change their stat to 'smooth' or 'density', nothing happens, the transformation doesn't take place and you get back the original plot!

[MarcSkovMadsen]

Hey @MarcSkovMadsen 👋🏼, upon experimenting a bit with the work around you provided, it seems to break down when using plots having their stat parameter not defaulted to 'identity'. If you try to plot geom_density() or geom_smooth() for example, it spits out an error because lets-plot starts looking for the dataframe's meta data such as ..quantile.. and ..ymin/max.. for geom_density() and geom_smooth(), respectively.

But even upon manually providing those variables by augmenting the dataframe before feeding it into ggplot(), the rendered plots just reduce to something that you would get from geom_line().

Also, for any of the ordinary plots with their stat parameter defaulted to 'identity', if you change their stat to 'smooth' or 'density', nothing happens, the transformation doesn't take place and you get back the original plot!

Hi @OSuwaidi . Could you provide a minimum, reproducible example? That would make it very easy for me or someone else to start root cause analysis. Thanks.

[OSuwaidi]

Sure!
We first create the custom lets-plot pane (I used polars over pandas because plot wouldn't even render with pandas):

import param
from panel.reactive import ReactiveHTML

from lets_plot import __version__ as _lets_plot_version
from lets_plot.plot.core import PlotSpec

import polars as pl


class LetsPlotPane(ReactiveHTML):
    object: PlotSpec = param.ClassSelector(class_=PlotSpec, precedence=-1)

    _plot_spec_as_dict = param.Dict()

    _template = '<div id="pn-container" style="height:100%;width:100%"></div>'

    __javascript__ = [
        f"https://cdn.jsdelivr.net/gh/JetBrains/lets-plot@v{_lets_plot_version}/js-package/distr/lets-plot.min.js"
    ]

    @param.depends("object", watch=True, on_init=True)
    def _update_config(self):
        if not self.object:
            self._plot_spec_as_dict = {}
        else:
            spec = self.object.as_dict()
            if "data" in spec and isinstance(spec["data"], pl.DataFrame):
                spec["data"] = spec["data"].to_dict(as_series=False)

            self._plot_spec_as_dict = spec

    _scripts = {
        "render": "state.height=-10",
        "after_layout": "self._plot_spec_as_dict()",
        "_plot_spec_as_dict": """
var height=pn_container.clientHeight
if (state.height-5<=height & height<=state.height+5){height=state.height}
state.height=height
pn_container.innerHTML=""
LetsPlot.buildPlotFromProcessedSpecs(data._plot_spec_as_dict, pn_container.clientWidth-5, height, pn_container);
""",
    }

Below is sample data:

np.random.seed(0)
sample_revenue = np.random.randint(1, 10000, size=100)
sample_time = range(100)

df = pl.DataFrame({
    'revenue': sample_revenue,
    'time': sample_time
})
p = (ggplot(df.with_columns(pl.min('revenue').alias('..ymin..'), pl.max('revenue').alias('..ymax..')))
 + geom_smooth(aes(x='time', y='revenue'), method='loess', span=0.43, size=1.5)
  )
LetsPlotPane(object=p).servable()

Result:

Expected: