'dvc plots show' cannot plot using nested keys

[mattrmd]

I have a metrics.yaml file with the following structure:

- dataset_a:
    metric_1: ...
    metric_2: ...
  dataset_b:
    metric_1: ...
    metric_2: ...
...

I would like to use dvc plots show metrics.yaml -y dataset_a.metric_1 to plot the first metric. However, it seems that DVC is not capable of doing this, as I get the error:

Could not find provided field ('dataset_a.metric_1') in data fields ('dataset_a, dataset_b').

This appears to be a shortcoming in the vega.py file, specifically with the functions _verify_field, _get_xs, and _get_ys. There may be others as well, but I have not looked past these functions yet. It appears that these functions look only at the top level key.

I would like to try extending the functionality of dvc plots show to allow for nested keys. I think that it should be relatively straightforward to use an approach similar to this, but that remains to be seen. I will try to implement this change over the next few weekends, and hopefully my team will be able to use it in production.

[dberenbaum]

Is dataset_a.metric_1 an array or a scalar value? Can you give a more detailed example of the dataset and desired plot result?

Note that if you do dvc plots show metric.syaml -y metric_1, it should actually plot dataset_a.metric_1 since DVC parses the first array found in the file, but I'm not sure that's what you're looking for.

[mattrmd]

Is dataset_a.metric_1 an array or a scalar value? Can you give a more detailed example of the dataset and desired plot result?

It is a scalar. Here is an example of a metrics.yaml file that I could have, where each array element represents an epoch:

# metrics.yaml
- dataset_a:
    metric_1: 0
    metric_2: 1
  dataset_b:
    metric_1: 100
    metric_2: 100
- dataset_a:
    metric_1: 3
    metric_2: 4
  dataset_b:
    metric_1: 66
    metric_2: 21
- dataset_a:
    metric_1: 6
    metric_2: 21
  dataset_b:
    metric_1: 33
    metric_2: 4
- dataset_a:
    metric_1: 10
    metric_2: 100
  dataset_b:
    metric_1: 0
    metric_2: 1

This is what it looks like when I plot each dataset/metric combination independently:

If I were to load this dataset using pyyaml, I could get an array of each dataset/metric combination as y = [epoch['dataset_a']['metric_1'] for epoch in metrics], where metrics: dict[str,dict[str,float]].

I would like to be able to do something similar using DVC (but only plotting one line) by giving the command

$ dvc plots show metrics.yaml -y dataset_a.metric_1
ERROR: Could not find provided field ('dataset_a.metric_1') in data fields ('dataset_a, dataset_b').

Note that if you do dvc plots show metrics.yaml -y metric_1, it should actually plot dataset_a.metric_1 since DVC parses the first array found in the file, but I'm not sure that's what you're looking for.

If I run that command, I get the following:

$ dvc plots show metrics.yaml -y metric_1
ERROR: Could not find provided field ('metric_1') in data fields ('dataset_a, dataset_b').

I'm using Python 3.10.6 on Ubuntu 22.04.2, with the following DVC versions:

dvc==2.51.0
dvc-data==0.44.1
dvc-http==2.30.2
dvc-objects==0.21.1
dvc-render==0.3.1
dvc-studio-client==0.6.1
dvc-task==0.2.0

Based on the code that I read, I think that accessing nested dictionary values is not possible using the current implementation of DVC's dvc plot show command.

[dberenbaum]

Thanks, @mattrmd! Makes sense now. Sounds like a good improvement.

I would like to try extending the functionality of dvc plots show to allow for nested keys. I think that it should be relatively straightforward to use an approach similar to this, but that remains to be seen. I will try to implement this change over the next few weekends, and hopefully my team will be able to use it in production.

Let us know how it goes. You might want to take a look at https://funcy.readthedocs.io/en/stable/colls.html#has_path as an alternative to the stackoverflow link you mentioned.