Skip to content

[DF] Separate computation graph from dataset it should run on #15070

Open
@martamaja10

Description

@martamaja10

Explain what you would like to see improved and how.

Moving from JIRA: ROOT-9438

Advanced users (e.g. TMVA internals) might want to define a computation graph once (e.g. by jitting) and then apply it to several different data-sets or several subsets of a dataset in sequence, without having to re-generate the computation graph each time.
The introduction of a new type of head node, a "computation manager", which can attach to different "loop managers", would be one relatively simple way to allow this.
Common usage of RDataFrame would remain the same, with the loop manager using a computation manager to "speak" to the computation graph under the hood. Explicit usage of the computation manager node would allow to attach to different loop managers.
At least the schema of the dataset (or the subset that one expects to read) should probably still be known a priori, to allow RDataFrame to error out early in case of misspelled column name, inexistent branch name and so on.
To allow the reusage of the same computation graph, actions should not be deleted at the end of the event loop as it is currently done but kept around for the next run.

ROOT version

all

Installation method

all

Operating system

all

Additional context

No response

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions