feat: DH-18281 Input Filters UI Component

[dgodinez-dh]

Draft PR to discuss implementation.

[github-actions]

ui docs preview (Available for 14 days)

[mofojed]

This works, which is great, but I don't like having to mount the input_filters component to get this to work... if you want input filters in multiple components it's resending the same list multiple times on change. To do this right, I think we need to have the RenderContext itself manage an input filter state from the client. We already have a set_state function, I think we'd need a set_input_filters command... and would probably want the component to also notify with subscribe/unsubscribe so it doesn't send it unnecessarily...

The more I think of this, the more I want to break the problem down into two categories:

1. Existing one_click functionality: Getting the same functionality we have with Java charts and oneClick, where the user is prompted to add a filter before being able to view the chart.
2. Using the clients input filters from their dashboard within your deephaven.ui component (e.g. a use_dashboard_inputs hook, or as you've got here the input_filters component).

By doing 2, we can also do 1. Another advantage of 2 is that you don't need to change Deephaven Express or any other widgets, since it's re-rendering them entirely with a new table. On the downside, it's a little more syntax heavy, and it requires some architectural changes in how we manage deephaven.ui rendering (as we need to pass the client info along). That architectural change does open up interesting possibilities (e.g. themeing, formatting, etc)...

That all being said - I think we're straying away from the initial goal here, which is to just require some filters to be set on a table before attempting to plot it. My concern is that we're making this too complicated/confusing, when we really just need 1. Our main argument for avoiding 1 is that it would require changing Deephaven Express, every widget would need to build special support for it. But I think we could do 1 with minimal changes to existing one_clicks/deephaven express, and ultimately we'd be the better for it - we're re-using a lot of existing infrastructure. Then we can also do 2.

Right now a big problem with what's returned from the one_click API is that it's just a Java object, and you can't access any methods. If you could access the table definition (like you can from Java), then we'd be good. For Deephaven Express, all we really need is the schema/table definition on the Python side, the rest is applied client side. So if we accept anything with a TableDefinition (rather than requiring a whole table), then we are good for the most part.

And the SelectableDataSet that is returned from one_click does have that: https://github.com/deephaven/deephaven-core/blob/main/Plot/src/main/java/io/deephaven/plot/filters/SelectableDataSet.java
So we should be able to get that in Deephaven Express by calling toc.j_object.getTableDefinition(). Which I think the only spot we really need info from the table is generating the mappings here:

deephaven-plugins/plugins/plotly-express/src/deephaven/plot/express/deephaven_figure/generate.py

Line 162 in 01693d7

for col in table.columns:

So could just check if it's a selectable data set there and just get the table definition instead.

I think we should look at that first, as it may not actually be a lot of work to do 1 and then we can do 2 which leads to more interesting/powerful cases with deephaven.ui and custom widgets, but is more difficult to implement/more expensive.

[mofojed]

The columnNames shouldn't be changing, that's something I'll need to look into.

[mofojed]

So the widgetId is used in DashboardWidgetHandler, but isn't passed down. That could be passed down in a context - it would probably make sense to just pass it down to WidgetHandler and then WidgetHandler adds it to the WidgetStatus (which can then be accessed with useWidgetStatus).

[dgodinez-dh]

Yeah, the issue wasn't getting widget id. Originally, I was trying to get a ref to the panel. Then I started a refactor to have the event use panelId and you suggested widgetId:

https://deephaven.atlassian.net/browse/DH-19267

[mofojed]

@dgodinez-dh I have a branch modifying the Python side of Deephaven Express to allow passing in the SelectableDataSet.. there'd still need to changes to transport that SelectableDataSet object from the server to the client: 698d46c

Or ... we could do something like just wrap the Table object, and only return the underlying table with filters when the correct filters are applied, e.g.:

from deephaven import empty_table
from deephaven.table import Table
from deephaven.plot import express as dx
from deephaven.filters import Filter

class TableWrapper(Table):
    def __init__(self, table: Table):
        self._table = table
        self._blank_table = table.head(0)
    
    def __getattr__(self, attr):
        # Proxy all requests to the blank_table so as not to incur any expensive operations
        return getattr(self._blank_table, attr)
    
    def where(self, *args, **kwargs) -> Table:
        # TODO: If all the required filters are present, then return the original table but with the filters applied
        print("Applying table where", *args, **kwargs)
        return self._table.where(*args, **kwargs)

_stocks = dx.data.stocks()
_stocks_filterable = TableWrapper(_stocks)
p = dx.line(_stocks_filterable, "Timestamp", "Price")  # TODO: Need to filter the table correctly
p2 = dx.line(_stocks_filterable.where(["Sym=`CAT`", "Exchange=`TPET`"]), "Timestamp", "Price")
t3 = _stocks_filterable.where(["Sym=`CAT`", "Exchange=`TPET`"])

Though that gets sticky because the python Table is just a wrapper for the j_table, so we'd need to actually wrap the returned j_table with a Java proxy for most of the methods ...

[jnumainville]

My main concern, and hence pushback into building the logic into dx, is is for chart types that don't take the happy path of passing a Table into the chart and sending the same Table to the client. Or in this case taking a SelectableDataSet and passing that same SelectableDataSet to the client. There is tons of logic that transforms tables.
For example, a plot by in dx translates to a partition_by on the Table. I use that example because know that plot by one_click charts are desirable, but there are other charts that must be transformed on some or all paths, such as histogram.
We would need to proxy SelectableDataSet as a Table, then operate on that Table, so that the filter happens, then downstream transformations are applied (partitions, etc.). I see a getSwappableTable but that doesn't look to be the right API for this .
Unless that functionality exists and I am missing it, if we want full support in this way I think we would have to build a custom wrapper or use client to server bidirectional communication with different syntax so that the one_click is handled internally in dx, which seems more complicated than just having dh.ui handle it.