Improving the NRDS Predictions via Community Realization Contributions

[JordanLaserGit]

Copied from #170.

There is on going development of the tooling and workflows that will allow members of the hydrologic modeling community to propose NextGen parameters that will ideally improve the Research DataStream forecasts.

This process is split into two workflows:

1. One which is triggered by a community proposer via some interface. Google Forms? GitHub issue? I have problems with either so up for ideas here. This workflow should be responsible for
   a) Specifying the minimum files required to make a proposal (realization and hydrofabric).
   b) Validating these files in so far as consistency and expected structure. The current DataStreamCLI validation can be adapted for this purpose.
   c) Appending the parameters to a parameters.parquet object in s3.
2. One which is triggered by pushing a button to accept a proposed set of updated parameters. We will need an interface where the button is pushed. This workflow should be responsible for
   a) Updating the official parameters.parquet with the proposed parameters.
   b) Create realizations from the official paramaeters.parquet that contain the per-catachment updated parameters. These realizations are stored in s3 and are read by each research datastream execution (no local caching of nextgen configuration).

Thoughts about these workflows

• Accepted updated parameters will be used in the next and following research datastream executions. This means that the incoming updated parameters will overwrite the previous parameters.
• Proposing changes makes no impact on the research datastream configuration or performance.
• Proposing and accepting changes are decoupled.
• This process is unique to a given hydrofabric version.

Open questions to answer

• Where should the tooling and workflows exist? This repository already contains all related DataStreamCLI tooling and the infrastructure/tooling for the research datastream. We may want to create different repositories to separately manage parameters.

• Who pushes the button and how do they decide to push it? Consider the following: Proposer 1 successfully commits their parameters and feels very happy with their contributions. Proposer 2 comes along with parameters for the same catchments (or at least over lapping with the region of Proposer 1) and claims their parameters superior. If we accept Proposer 2's parameters, Proposer 1's contributions will be overwritten and they will reasonably want an explanation. What should we tell them?

  • Ideally this would be in the hands of an all seeing and all powerful wizard who would know the heart of the parameters and determine which will produce better streamflow forecasts within the context of the nextgen research datastream.
  • An automated evaluation workflow could supply baseline metrics by which new parameters can be analyzed (and the button pusher can subsequently discuss and investigate further). This evaluation process could be motivated by previous national scale evaluation studies (Mike's paper is great) and could serve as a standardized goal post (albeit an editable prototype) by which members of the community could gauge their parameters.

• Any that anyone else think of, please leave comments!

[JordanLaserGit]

Here's a potential form we can use. This at the least demonstrates what I'm trying to accomplish with the form/submission standard.

NextGen Research DataStream (NRDS) Community Parameters

The hydrologic simulations conducted by the NRDS are configured with the NextGen realization file. By default, NRDS will calculate the model parameters from the hydrofabric, which means the default parameters are uncalibrated.

In order to improve the accuracy of the streamflow predictions made by the NRDS, these parameters are open to the community and individual members may propose calibrated parameters to be used by the system in future executions.

If you have calibrated parameters and would like to submit them, you can do by preparing a data package that abides by the standard below.

Submission Standard

A submission data package contains 2 files.

├── hydrofabric_subset.gpkg		
├── realization.json				

1. The geopackage that holds the hydrofabric used during calibration.

NOTE: The hydrofabric is an evolving data product, with new version releases expected semi-annually. Calibrated parameters are only valid for a specific hydrofabric version. As of June 2025, submissions are only accepted with v2.2 hydrofabric.

2. A NextGen realization that holds the parameters and model selection.

   One of two types of realizations are allowed in the submission.

   1. Lumped - a single set of parameters are defined in the realization. These parameters will be applied to all catchment in the geopackage.
   2. Per-catchment - a set of parameters is defined for each catchment.

Tar up the data package into a single object. Store the object somewhere publicly and provide the object's URL while filling out the submission form here NEED LINK HERE.

Example tar command:

[jlaser@LYNK-59WW6S3 data]$ tar -czvf example_community_contribution.tar.gz example_community_contribution/
example_community_contribution/
example_community_contribution/gage-10154200_subset.gpkg
example_community_contribution/realization.json

Example data package URL:

https://ciroh-community-ngen-datastream.s3.us-east-1.amazonaws.com/submissions/example_community_contribution.tar.gz

[JordanLaserGit]

We can validate the realization's BMI variable mappings (motivation for doing so #205) using a similar approach as @aaraney does here. This script builds the model_stack from ngen-cal classes, where we would need to do this from the realization file. Also, models' variable names are hard coded. Ideally the models could self describe their variable names, but we could take this approach for now to provide some baseline validation capacity.

[arpita0911patel]

As per the data and code policies here - https://docs.ciroh.org/docs/policies/DataAndCodeSharingPolicy/Recommendations we should ask them to upload the tar bundle in hydroshare and provide us the link by opening a Issue ticket?

[JordanLaserGit]

Sure, hydroshare is a good option. It will provide a DOI so that the contributions can be referenced later.

[JordanLaserGit]

Motivation for validation of realizations - > https://github.com/CIROH-UA/ngen-datastream/issues/200#issuecomment-3180788304. We may want some options set in the realization for performance reasons.