Skip to content

Improve Usability of Eval Outputs #508

Description

@adam-subel

Currently the outputs generated from the eval script create unnecessary friction when performing analysis, and may lead to unintended errors. I am including an example notebook below, which highlights several ways in which the analysis process is hindered.

  1. The data is separated by level for ease of ML training, however, given that the eval outputs are not used in this context, we introduce the step of concatenating variables back to their respective depths, while this is not a big lift, and mistakes in doing this could an additional place for errors.

  2. The horizontal coordinates of the eval outputs do not align with the true data (['lat','lon'] vs ['y','x']) this is unnecessary and requires manually renaming axes

  3. We are missing most of the grid information necessary for performing calculations. At the moment, I address this by recomputing them from the same grid files used in processing the data; however, for future proofing, I think that this grid information should either live in the eval datasets, or we should create a grid zarr for each resolution and store a path to that grid in the attrs. These differences, have an impact on conservation as slight changes to depth, area, or lat/lon locations can make a difference

Sample_Eval_Notebook_OM4.ipynb

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions