|
| 1 | +<!-- |
| 2 | +Copyright 2025 The DAPHNE Consortium |
| 3 | +
|
| 4 | +Licensed under the Apache License, Version 2.0 (the "License"); |
| 5 | +you may not use this file except in compliance with the License. |
| 6 | +You may obtain a copy of the License at |
| 7 | +
|
| 8 | + http://www.apache.org/licenses/LICENSE-2.0 |
| 9 | +
|
| 10 | +Unless required by applicable law or agreed to in writing, software |
| 11 | +distributed under the License is distributed on an "AS IS" BASIS, |
| 12 | +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 13 | +See the License for the specific language governing permissions and |
| 14 | +limitations under the License. |
| 15 | +--> |
| 16 | + |
| 17 | +# Recording Data Properties |
| 18 | + |
| 19 | +There is an experimental feature for recording the true data properties of intermediate results at run-time and re-inserting them in a subsequent run of DAPHNE. |
| 20 | + |
| 21 | +By default, this feature is turned off. |
| 22 | +Recording data properties can be turned on by invoking `daphne` with `--enable-property-recording`. |
| 23 | +Inserting data properties recorded in a previous execution of `daphne` can be turned on with `--enable-property-insert`. |
| 24 | +These two options are mutually exclusive. |
| 25 | +The recorded true data properties are stored to/loaded from a simple JSON file whose path is specified by `--properties-file-path` (default: `properties.json`). |
| 26 | +See also `daphne --help` for some help on these options. |
| 27 | + |
| 28 | +## Possible Uses of this Feature |
| 29 | + |
| 30 | +- Recording: Detailed insights into the data properties of intermediate results to find potential for optimizations by exploiting these data properties. |
| 31 | +- Insertion: Find out how DAPHNE performs when it has precise knowledge of the true data properties (no unknowns, no inaccurate compile-time estimates). |
| 32 | +- ... |
| 33 | + |
| 34 | +## Current Limitations |
| 35 | + |
| 36 | +- Only one data property is recorded: sparsity; more properties could be added in the future. |
| 37 | +- Only matrix-typed intermediate results are considered; scalars and frames could be added in the future. |
| 38 | +- Only intermediates produced in the `main` function of DaphneIR are considered; intermediates inside UDFs could be considered in the future. |
| 39 | +- Control-flow constructs like if-then-else and loops are viewed as black boxes, i.e., the data properties of their results are recorded/inserted, but not those of intermediates created *inside* these constructs; in the future, we could consider those, too. |
| 40 | +- ... |
0 commit comments