WeatherQuest Calibration pipeline with weekly support by costachris · Pull Request #1747 · CliMA/ClimaCoupler.jl

costachris · 2026-02-20T23:33:32Z

Purpose

To-do

Content

I have read and checked the items on the review checklist.

diable O3 timevarying, land periodic cal LAI Use MODIS LAI climatology (land) and albedo from era5 (bucket) for historical runs. Initialize sea ice with sea ice temperatures from era5. Fix sea ice (use surface temp for init). Use config outputs consistent with era5 val comparison. lower ice min temp and revert CL grid update config Remove oceananigans and climaocean. Pin GPU compilers. Add subseasonal calibration pipeline Update calibration pipeline for weatherquest weekly calibrations on derecho Add and use 1 day calibration. Fix some bugs in observations map and daily data handling. add data normalizations and modify noise. Add land mask, modify noise, add land parameter, write to scratch and deal with HDF errs Add support and functionality for TransformInversion, Inversion. Precompute large matrices on cpu as inital step add gravity wave to calibration, move to copies3 add options, add more parameters, log top level run script fix normalization bug for multi variable option. Add ice albedo to toml. fix precip units, add logging, start trying 7 day weekly runs Add NH average option, remove precip, all specifically of noise by var for NH option, add ensemble spatial plotting update plots Add script for analyzing parameters clean rebase update manifests add TOA flux, update prior

…week comparison period to 1 month CERES data for now.

costachris · 2026-02-23T22:05:27Z

experiments/calibration/subseasonal_weekly/run_full_calibration.pbs

@@ -0,0 +1,53 @@
+#!/bin/bash


will likely remove this file (in favor of run_full_calibration.sh‎)

costachris · 2026-02-23T22:10:12Z

experiments/calibration/subseasonal_weekly/run_calibration.jl

+    sample_date_ranges,
+    # Monthly run: 7-day spinup + 21-day calibration = 28 days total
+    # Model starts at (Jan 8 - 7 days) = Jan 1, ends at (Jan 8 + 21 days) = Jan 29
+    extend = Dates.Day(21),


This is a temp hack to compare 3 weeks of model data (after 1 week spinup) to 1 month of CERES data.
I'm adding IC files starting a week before the start date, so then we can probably do the 1 week of spinup + 1 month of comparison.

szy21 · 2026-02-23T23:22:36Z

experiments/calibration/subseasonal_weekly/generate_observations.jl

+    else
+        # Compute from h_elem (spectral element grid)
+        h_elem = get(config_dict, "h_elem", 12)
+        # Default formula: h_elem * 4 panels * 3 (spectral degree)
+        nlon = h_elem * 4 * 3
+        nlat = nlon ÷ 2
+        @info "Using model grid from h_elem=$h_elem: $(nlon)×$(nlat)"
+    end


use the actual model output grid

ph-kev · 2026-02-23T23:04:34Z

experiments/calibration/subseasonal_weekly/observation_map.jl

+end
+
+"""
+    time_average_with_date(var, date)


I can add a function for adding a singleton dimension in ClimaAnalysis, so you don't need to do this.

ph-kev · 2026-02-23T23:06:18Z

experiments/calibration/subseasonal_weekly/observation_map.jl

+        end
+    end
+
+    g_ensemble = EnsembleBuilder.get_g_ensemble(g_ens_builder)


This pattern should not be followed since you don't know if the G ensemble matrix is fully completed. See

ClimaCoupler.jl/experiments/calibration/subseasonal/observation_map.jl

Lines 39 to 44 in ff10c2e

g_ens = EnsembleBuilder.get_g_ensemble(g_ens_builder)

if count(isnan, g_ens) > 0.9 * length(g_ens)

error("Too many NaNs")

end

return EnsembleBuilder.is_complete(g_ens_builder) ? g_ens :

error("G ensemble matrix is not completed")

ph-kev · 2026-02-23T23:13:39Z

experiments/calibration/subseasonal_weekly/generate_observations.jl

+    covar_estimator = ClimaCalibrate.ObservationRecipe.ScalarCovariance(;
+        scalar = Float64(CALIBRATION_NOISE_SCALAR),
+        use_latitude_weights = true,
+    )


I realized this function doesn't work too well since sample_date_ranges is overloaded for both calibration and what samples to choose for the observation. For example, the samples might be over 50 years, but you are only running a calibration for a couple of years.

ph-kev · 2026-02-23T23:29:34Z

experiments/calibration/subseasonal/observation_utils.jl

+    # Surface temperature/pressure
    "pr" => "kg m^-2 s^-1",


This should be simplified once we put all of this in a data loader.

ph-kev · 2026-02-23T23:30:20Z

experiments/calibration/subseasonal/observation_utils.jl

+
+Shifts the time axis by the largest period in the date range, sets units, and
+windows to the date range. Calls `largest_period` (defined per-pipeline) to
+determine the shift.


There shouldn't be a need for shifting the times.

ph-kev · 2026-02-23T23:35:41Z

experiments/calibration/subseasonal_weekly/generate_observations.jl

+
+
+# Lazy-loaded CERES data loader (initialized on first use)
+const _CERES_LOADER = Ref{Union{Nothing, CalibrationTools.CERESDataLoader}}(nothing)


Creating a CERESDataLoader shouldn't be that expensive since it only involves opening the NetCDF file and parsing the metadata it.

ph-kev · 2026-02-23T23:36:56Z

experiments/calibration/subseasonal_weekly/generate_observations.jl

+    var = ClimaAnalysis.window(var, "time"; left = month_start, right = month_end)
+
+    # Get the data for this month (should be single time point)
+    times = ClimaAnalysis.times(var)
+    if length(times) == 0
+        error("No CERES data found for $short_name in month of $start_date")
+    end


By default, window always get the nearest dates. I don't think it is possible for the length of times to be zero.

ph-kev · 2026-02-23T23:37:22Z

experiments/calibration/subseasonal_weekly/generate_observations.jl

+    if length(times) > 1
+        var = ClimaAnalysis.average_time(var)
+    end


I don't like this since it is introducing a fallback that shouldn't exist.

ph-kev · 2026-02-23T23:39:57Z

experiments/calibration/subseasonal_weekly/generate_observations.jl

+    # CERES dates are at start of month, so window to get the correct month
+    var = ClimaAnalysis.window(var, "time"; left = month_start, right = month_end)


Instead of this, this should use selectwith MatchValue().

ph-kev · 2026-02-23T23:43:11Z

experiments/calibration/subseasonal_weekly/generate_observations.jl

+Compute global mean and std for each variable across all date ranges.
+Returns a Dict mapping short_name -> (mean, std).
+Uses latitude-weighted averaging for physically meaningful statistics.
+"""


This can probably be implemented in ClimaCalibrate or ClimaAnalysis instead.

costachris added 5 commits February 21, 2026 15:02

restore subseasonal dir to match main

5778e99

add subseasonal weekly folder

c090c81

remove NH mask

a090e74

consolidate weekly and monthly

7011dfb

costachris force-pushed the cc/wxquest_v4_final branch 2 times, most recently from 52210eb to f00776f Compare February 21, 2026 23:41

remove unused files.

5ce743c

costachris force-pushed the cc/wxquest_v4_final branch from f00776f to 5ce743c Compare February 22, 2026 01:20

consolidate model interface.

fbfddca

costachris force-pushed the cc/wxquest_v4_final branch from 61e2107 to fbfddca Compare February 22, 2026 19:34

Add back variable normalization. Setup for derecho.

f353f7f

costachris changed the title ~~Cc/wxquest v4 final~~ WeatherQuest Calibration pipeline with weekly support Feb 22, 2026

costachris force-pushed the cc/wxquest_v4_final branch from b2ddbf9 to b9f1b27 Compare February 23, 2026 04:32

clean up comments

6b522e6

costachris requested review from nefrathenrici and ph-kev February 23, 2026 18:22

add CERES dataloader.

96a7399

costachris force-pushed the cc/wxquest_v4_final branch from b9f1b27 to 96a7399 Compare February 23, 2026 20:07

Add CERES data support to pipeline. Use extend + spinup to compare 3 …

9a68101

…week comparison period to 1 month CERES data for now.

costachris commented Feb 23, 2026

View reviewed changes

average daily output files in obs map to get aggregated values for loss.

fec3460

szy21 reviewed Feb 23, 2026

View reviewed changes

ph-kev reviewed Feb 23, 2026

View reviewed changes

	g_ens = EnsembleBuilder.get_g_ensemble(g_ens_builder)
	if count(isnan, g_ens) > 0.9 * length(g_ens)
	error("Too many NaNs")
	end
	return EnsembleBuilder.is_complete(g_ens_builder) ? g_ens :
	error("G ensemble matrix is not completed")



		# Lazy-loaded CERES data loader (initialized on first use)
		const _CERES_LOADER = Ref{Union{Nothing, CalibrationTools.CERESDataLoader}}(nothing)

		# CERES dates are at start of month, so window to get the correct month
		var = ClimaAnalysis.window(var, "time"; left = month_start, right = month_end)

Comments

Conversation

costachris commented Feb 20, 2026

Purpose

To-do

Content

Uh oh!

costachris Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

costachris Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

costachris Feb 23, 2026 •

edited

Loading

costachris Feb 23, 2026 •

edited

Loading