Add GEnsembleBuilder by ph-kev · Pull Request #222 · CliMA/ClimaCalibrate.jl

ph-kev · 2025-08-22T23:55:46Z

closes #221, closes #211, closes #220

This PR adds GEnsembleBuilder to facilitate easy construction of the G ensemble matrix using the metadata in the observation.

ph-kev · 2025-08-26T20:03:41Z

ext/ensemble_builder.jl

+    pkgversion(ClimaAnalysis) > v"0.5.19" || error(
+        "Using GEnsembleBuilder requires a version of ClimaAnalysis above 0.5.19",


I don't know if we want to continue supporting ClimaAnalysis v0.5.19 and below, since there will probably need to be a change to support passing flatten keyword arguments when constructing the covariance matrix.

I think we should only support >0.5.19, I don't think we have users that want to use old versions of ClimaAnalysis with this package anyway.

ext/ensemble_builder.jl

ph-kev · 2025-08-26T20:08:46Z

ext/ensemble_builder.jl

+function _is_compatible_with_metadata(var::OutputVar, metadata::Metadata)
+    return _same_short_names(var, metadata) &&
+           _same_dim_names(var, metadata) &&
+           _same_dim_units(var, metadata) &&
+           _same_units(var, metadata) &&
+           # For the temporal dimension, only check if the times of metadata is
+           # a subset of the times of var, because _match_dates will get the
+           # correct dates for us
+           _compatible_dims_values(var, metadata)
+end


This need to be changed, since it is not very helpful to the user right now. For example, if a OutputVar doesn't match with any of the metadata, it would help if there was logging that tell you why it can't match with the metadata that was tried. Instead, it tells you that none of the metadata the function tried match with the OutputVar. This could be resolved with a verbose keyword argument or another function that can be used to diagnose the issues. Furthermore, I would like this system to be extensible, where the user can pass in their own custom checks.

I was thinking of doing it in another PR to avoid this one to become more bloated than it is right now.

I agree, this will be an important addition in a future PR.

ph-kev · 2025-08-26T20:11:01Z

I think this is good enough for a review, but not ready to merge (still need to release a ClimaAnalysis version and missing/helpful features for building G ensemble matrix).

ph-kev · 2025-08-26T21:49:14Z

docs/src/ensemble_builder.md

+!!! info "Spinup and windowing times"
+    Internally, the correct dates are matched between the observational and
+    simulation data. As a result, you do not need to window the times (e.g. when
+    removing spinup) to match the times of the observations.


I am not too sure about this, since it could introduce correctness issues that might be overlooked. Although, it is convenient to automatically just match times, so you don't need to worry about windowing or slicing the time dimension.

ph-kev · 2025-08-27T00:17:14Z

ext/ensemble_builder.jl

+    ::Type{FT},
+    ekp::EKP.EnsembleKalmanProcess,
+) where {FT <: AbstractFloat}
+    pkgversion(ClimaAnalysis) > v"0.5.19" || error(


Make sure to add a check for EKP as well

The plan is to use the latest version of EnsembleKalmanProcesses and ClimaAnalysis, so this is not necessary.

nefrathenrici

This looks great so far! I think we can push this a bit further and make more information easily available to the user.

src/ekp_interface.jl

nefrathenrici · 2025-08-28T18:44:17Z

ext/ensemble_builder.jl

+    pkgversion(ClimaAnalysis) > v"0.5.19" || error(
+        "Using GEnsembleBuilder requires a version of ClimaAnalysis above 0.5.19",


I think we should only support >0.5.19, I don't think we have users that want to use old versions of ClimaAnalysis with this package anyway.

ext/ClimaAnalysisExt.jl

ext/ensemble_builder.jl

ph-kev · 2025-09-02T21:30:51Z

The _match_dates function can be more general and be moved to ClimaAnalysis as select which is a function that allows for arbitrary indexing of dimensions by index or value. See this issue CliMA/ClimaAnalysis.jl#323.

ph-kev · 2025-09-03T20:43:50Z

I think this is ready for another review. The only change that I want to make after this is maybe looking over how the functions are named and making another function in ClimaAnalysis to replace most of _match_dates.

nefrathenrici

I think this is ready to merge, I just left some small comments that we can discuss later today.

docs/make.jl

nefrathenrici · 2025-09-03T21:26:27Z

ext/ClimaAnalysisExt.jl

+        allunique(ClimaAnalysis.dates(var)) || @warn(
+            "Dates in OutputVar with short name $(short_name(var)) are not unique. You will not be able to use GEnsembleBuilder",
+        )
+    end
+
+    any(==(""), ClimaAnalysis.short_name.(vars)) && @warn(


Should these just be warnings or actual errors?

We discussed this offline, but these should be warnings because an user might not want to use GEnsembleBuilder for whatever reason.

ext/ClimaAnalysisExt.jl

ext/ensemble_builder.jl

nefrathenrici · 2025-09-03T21:30:29Z

ext/ensemble_builder.jl

+function _is_compatible_with_metadata(var::OutputVar, metadata::Metadata)
+    return _same_short_names(var, metadata) &&
+           _same_dim_names(var, metadata) &&
+           _same_dim_units(var, metadata) &&
+           _same_units(var, metadata) &&
+           # For the temporal dimension, only check if the times of metadata is
+           # a subset of the times of var, because _match_dates will get the
+           # correct dates for us
+           _compatible_dims_values(var, metadata)
+end


I agree, this will be an important addition in a future PR.

src/ekp_interface.jl

ph-kev · 2025-09-08T21:01:07Z

docs/src/api.md

+ClimaCalibrate.EnsembleBuilder.is_complete
+ClimaCalibrate.EnsembleBuilder.get_g_ensemble
+ClimaCalibrate.EnsembleBuilder.ranges_by_short_name


For is_complete, there should be a variant of this for checking a single column. This is helpful since if it is not completed for the first column, it probably won't be completed for any of the columns.

ext/ensemble_builder.jl

The function used to be private, but with the release of v0.5.20, the function is public and should be used instead.

Checks for unique dates and short name are added to ensure that `GEnsembleBuilder` with the observations.

ph-kev force-pushed the kp/g_ens_col branch 4 times, most recently from 7446005 to 770f21d Compare August 26, 2025 20:01

ph-kev commented Aug 26, 2025

View reviewed changes

ph-kev requested a review from nefrathenrici August 26, 2025 20:12

ph-kev commented Aug 26, 2025

View reviewed changes

ph-kev commented Aug 27, 2025

View reviewed changes

nefrathenrici reviewed Aug 28, 2025

View reviewed changes

ph-kev commented Aug 28, 2025

View reviewed changes

ext/ensemble_builder.jl Show resolved Hide resolved

ph-kev force-pushed the kp/g_ens_col branch from 770f21d to 868c503 Compare September 2, 2025 18:48

ph-kev force-pushed the kp/g_ens_col branch 2 times, most recently from 48d73fb to 3a44f20 Compare September 3, 2025 20:40

ph-kev force-pushed the kp/g_ens_col branch from 3a44f20 to 421405b Compare September 3, 2025 20:45

ph-kev requested a review from nefrathenrici September 3, 2025 20:46

nefrathenrici approved these changes Sep 3, 2025

View reviewed changes

ph-kev commented Sep 8, 2025

View reviewed changes

ext/ensemble_builder.jl Outdated Show resolved Hide resolved

ph-kev force-pushed the kp/g_ens_col branch 4 times, most recently from cfc0329 to 6ecc0b9 Compare September 10, 2025 23:13

ph-kev mentioned this pull request Sep 10, 2025

Release v0.5.20 CliMA/ClimaAnalysis.jl#331

Merged

ph-kev force-pushed the kp/g_ens_col branch 4 times, most recently from 3de3183 to a3f6824 Compare September 11, 2025 21:12

ph-kev force-pushed the kp/g_ens_col branch 4 times, most recently from e797c55 to 741d576 Compare September 11, 2025 23:28

ph-kev added 10 commits September 16, 2025 10:46

Add g_ens_mat

836e2c0

Add get_minibatch_indices

2c710a1

Use ClimaAnalysis.flattened_length

d9e1b9e

The function used to be private, but with the release of v0.5.20, the function is public and should be used instead.

Fix typos in observation recipe docs

9b9c199

Fix broken tests for eltype

98c72ea

Add EnsembleBuilder

746b7ea

Add utilities functions for GEnsembleBuilder

792dc7c

Add show method for GEnsembleBuilder

c22f2bb

Add fill_g_ens_col! with scalar

96eac0a

Add checks when making observations

9b60ead

Checks for unique dates and short name are added to ensure that `GEnsembleBuilder` with the observations.

ph-kev force-pushed the kp/g_ens_col branch 2 times, most recently from dd617d6 to d980e59 Compare September 18, 2025 23:12

Update compat

f65ff58

ph-kev force-pushed the kp/g_ens_col branch 3 times, most recently from e880a55 to 8446d54 Compare September 19, 2025 17:09

Fix plotting in test

f9ac127

ph-kev force-pushed the kp/g_ens_col branch from 8446d54 to f9ac127 Compare September 19, 2025 17:55

ph-kev merged commit 47215aa into main Sep 19, 2025
7 of 9 checks passed

ph-kev mentioned this pull request Nov 14, 2025

Improvements to calibration pipeline #202

Open

		pkgversion(ClimaAnalysis) > v"0.5.19" \|\| error(
		"Using GEnsembleBuilder requires a version of ClimaAnalysis above 0.5.19",

Conversation

ph-kev commented Aug 22, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ph-kev Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ph-kev commented Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nefrathenrici left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ph-kev commented Sep 2, 2025

Uh oh!

ph-kev commented Sep 3, 2025

Uh oh!

nefrathenrici left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ph-kev Aug 26, 2025 •

edited

Loading

ph-kev commented Aug 26, 2025 •

edited

Loading