How can I manipulate Tables using complicated Pandas operations (rolling(), median()...)?

[jmy48]

I have what should be a very common use case, that is, ingesting streaming financial data, and computing indicators on-the-fly based off of the streaming price/volume data. I feel if, every tick, I simply collect into a DataFrame and then serialize into a Table to send it over the network, I'm not getting much out of Perspective. What should I do in this case?

Somewhat unrelated, but I also want to keep price/volume data for a set of tickers over a time axis, which would be 3D table (tensor). What's the best way to implement this with the Table API? A Table per ticker?

[texodus]

Perspective works a.little differently than Pandas' DataFrame currently, and certainly has fewer operations currently available. In 1.0.1, a <perspective-viewer> will only calculate aggregates when at least 1 "Group By"/row_pivot is selected, in which case it will also show a TOTAL row with the grand aggregate of all rows, as well as display the individual column's aggregate calculation selector. There is no way to enable the TOTAL row without such a grouping applied, but with ExprTK it is easy to add a Constant Column to any data set and group by this, if your dataset has no relevant column to group by, e.g.

viewer.restore({
    columns: ["Column I want median of"],
    aggregate: {"Column I want median of": "median"},
    expressions: ["1"],
    row_pivots: ["1"],
});

In many e.g. blotter-esque use cases we originally developer Perspective for, common group-by columns could be Ticker, Venue, Trade Type/Side, CounterParty, Trader/SalesPerson/Book, etc., so if I understand your second question correctly, the best way to represent this in Perspective is probably with a column "Ticker" per row that indicates the Ticker the price/volume observation represents, then use this field as a "Group By"/row_pivot (for calculating aggregates) or a "Split By"/column_pivot (for e.g. creating individual "Y Line" lines per ticker, when the "Group By" field is set to "Timestamp" or perhaps the expression bucket("Timestamp", 's') or similar). E.g.

viewer.restore({
    plugin: "Y Line",
    columns: ["volume"],
    row_pivots: ["bucket(\"timestamp\",'s')"],
    column_pivots: ["ticker"],
    expressions: ["bucket(\"timestamp\",'s')"],
});

The streaming example is very simple approximation of this pattern, and has "side", "client", "trader" and "ticker" fields; there are also a few more examples in the examples/ directory that make use of this fake blotter data. You may also find it useful to use <perspective-workspace> component to display multiple <perspective-viewer> elements with the same blotter Table efficiently; documentation for this feature is not quite ready but there are several examples showing basic usage e.g. movies and superstore.

There is no rolling aggregate, nor ExprTK window() function which could emulate it just yet, but these features are under development.