diff --git a/docs/CALLMIP_QUICKSTART.md b/docs/CALLMIP_QUICKSTART.md new file mode 100644 index 00000000000..e771997f8fa --- /dev/null +++ b/docs/CALLMIP_QUICKSTART.md @@ -0,0 +1,248 @@ +# CalMIP Quick Start Guide + +## Summary + +This guide provides a quick reference for setting up CalMIP sites in ClimaLand.jl. + +## Required Files for Each Site + +| File Type | Location | Purpose | +|-----------|----------|---------| +| CSV forcing data | `data/callmip_sites/SITE-ID.csv` | Meteorological forcing | +| Site configuration | `ext/fluxnet_simulations/SITE-ID.jl` | Site parameters | +| Simulation runner | `experiments/integrated/fluxnet/callmip_SITE-ID.jl` | Run script | + +## Climate Variables Required (CSV columns) + +``` +TIMESTAMP_START,TIMESTAMP_END,TA_F,VPD_F,PA_F,P_F,WS_F,LW_IN_F,SW_IN_F,CO2_F_MDS +``` + +| Variable | Description | Units | +|----------|-------------|-------| +| `TIMESTAMP_START` | Start time (YYYYMMDDHHMM) | - | +| `TIMESTAMP_END` | End time (YYYYMMDDHHMM) | - | +| `TA_F` | Air temperature | °C | +| `VPD_F` | Vapor pressure deficit | hPa | +| `PA_F` | Air pressure | kPa | +| `P_F` | Precipitation | mm | +| `WS_F` | Wind speed | m/s | +| `LW_IN_F` | Longwave radiation (down) | W/m² | +| `SW_IN_F` | Shortwave radiation (down) | W/m² | +| `CO2_F_MDS` | CO2 concentration | μmol/mol | + +## Quick Setup Steps + +### 0. Prepare CalMIP forcing data + +CalMIP CSV data files are NOT stored in git. Download from CalMIP and set up access: + +```bash +# Option 1: Use environment variable (recommended) +export CALLMIP_DATA_PATH="/path/to/your/callmip/data" +mkdir -p $CALLMIP_DATA_PATH + +# Option 2: Use default local path +mkdir -p ../callmip_data + +# Download and place your CSV files there +``` + +### 1. Update Artifacts.jl (if adding new sites) +```julia +# Add to: src/Artifacts.jl +function callmip_data_path(site_ID; context = nothing) + @assert site_ID ∈ ("DK-Sor", "YOUR-SITE") # ← Add here (only if adding new sites) + # ... +end +``` + +### 2. Create site config file +```bash +# Copy template: +cp ext/fluxnet_simulations/DK-Sor.jl ext/fluxnet_simulations/YOUR-SITE.jl +# Edit YOUR-SITE.jl with your site's parameters +``` + +### 3. Update FluxnetSimulationsExt.jl +```julia +# Add to: ext/FluxnetSimulationsExt.jl +include("fluxnet_simulations/YOUR-SITE.jl") # ← Add here +``` + +### 4. Create simulation runner +```bash +# Copy template: +cp experiments/integrated/fluxnet/callmip_dksor.jl \ + experiments/integrated/fluxnet/callmip_yoursite.jl +# Update site_ID and dates in callmip_yoursite.jl +``` + +### 5. Run simulation +```bash +# Make sure CALLMIP_DATA_PATH is set or data is in ../callmip_data/ +export CALLMIP_DATA_PATH="/path/to/callmip/data" + +julia --project=experiments experiments/integrated/fluxnet/callmip_yoursite.jl +``` + +## Site Information Needed + +For the site configuration file, you need: + +**Location:** +- Latitude (decimal degrees) +- Longitude (decimal degrees) +- Time zone offset from UTC (hours) + +**Tower/Measurements:** +- Measurement height (m) + +**Domain:** +- Soil depth (m, typically 10m or deeper) +- Vertical discretization (typically 20 elements) + +**Soil Properties:** +- Porosity (ν) +- Saturated hydraulic conductivity (K_sat) +- Van Genuchten parameters (α, n) +- Residual water content (θ_r) +- Soil texture fractions + +**Vegetation:** +- Canopy height (m) +- Stem height (m) +- Leaf height (m) +- Rooting depth (m) +- Plant functional type +- Typical LAI range + +## Example Site: DK-Sor + +```julia +# Site ID +site_ID = "DK-Sor" + +# Location (Denmark) +lat = 55.486°N +long = 11.6446°E +time_offset = +1 # CET (UTC+1) + +# Tower +atmos_h = 57 m + +# Vegetation +- Type: Deciduous Broadleaf Forest (Beech) +- h_canopy = 30 m +- h_stem = 25 m +- h_leaf = 5 m +- rooting_depth = 1.0 m +``` + +## CalMIP Data Sources + +1. **Protocol & Documentation:** + - https://github.com/callmip-org/Phase1 + +2. **Forcing Data:** + - Model Evaluation Portal: https://modelevaluation.org + - CalMIP GitHub: https://github.com/callmip-org/Phase1/tree/main/Data + +3. **Observation Data (for calibration):** + - https://github.com/callmip-org/Phase1/tree/main/Data + +## File Naming Convention + +**Use hyphens in file names and site_ID strings:** +```julia +site_ID = "DK-Sor" # ✓ Correct +site_ID = "DK_Sor" # ✗ Wrong +``` + +**Use underscores in Val types:** +```julia +Val(:DK_Sor) # ✓ Correct +Val(:DK-Sor) # ✗ Wrong +``` + +## Typical Parameter Values + +### For Deciduous Broadleaf Forests + +```julia +# Soil +soil_ν = 0.45 - 0.55 # Porosity +soil_K_sat = 1e-7 - 1e-6 # Hydraulic conductivity (m/s) +soil_vg_n = 1.5 - 2.5 # Van Genuchten n +soil_vg_α = 0.02 - 0.08 # Van Genuchten α (1/m) + +# Canopy +Vcmax25 = 5e-5 - 8e-5 # Max carboxylation rate (mol/m²/s) +g1 = 120 - 180 # Medlyn g1 parameter +h_canopy = 15 - 35 # Canopy height (m) +rooting_depth = 0.5 - 2.0 # Root depth (m) +``` + +### For Grasslands + +```julia +# Soil (similar ranges) +soil_ν = 0.40 - 0.50 +soil_K_sat = 5e-7 - 5e-6 + +# Canopy +Vcmax25 = 4e-5 - 7e-5 +g1 = 80 - 120 +h_canopy = 0.3 - 1.0 # Much shorter +rooting_depth = 0.3 - 1.0 # Shallower +``` + +## Checking Your Setup + +Before running a full simulation, verify: + +```bash +# 1. Check CSV file exists and has correct format +head data/callmip_sites/YOUR-SITE.csv + +# 2. Check Julia can load the site +julia --project=experiments -e ' +using ClimaLand +site_ID = "YOUR-SITE" +site_val = Symbol(replace(site_ID, "-" => "_")) +println("Site: ", site_val) +' + +# 3. Do a short test run (1 day) +# Edit your runner script to set: +# stop_date = start_date + Day(1) +``` + +## Common Errors + +### "Site not found in assertion" +→ Add your site to `callmip_data_path` in `Artifacts.jl` + +### "Cannot read CSV file" +→ Check path: `data/callmip_sites/YOUR-SITE.csv` + +### "Val type not defined" +→ Include your site file in `FluxnetSimulationsExt.jl` + +### "Method not found for Val{:YOUR_SITE}" +→ Use underscores in Val: `Val{:YOUR_SITE}` not `Val{:YOUR-SITE}` + +## Next Steps + +1. ✓ Complete setup (follow steps above) +2. Run test simulation (1-7 days) +3. Compare with observations +4. Set up calibration workflow +5. Follow CalMIP protocol for full runs + +## Additional Resources + +- Full documentation: `docs/CALLMIP_SETUP.md` +- Example site: `ext/fluxnet_simulations/DK-Sor.jl` +- Example runner: `experiments/integrated/fluxnet/callmip_dksor.jl` +- CalMIP website: https://callmip-org.github.io diff --git a/docs/CALLMIP_SETUP.md b/docs/CALLMIP_SETUP.md new file mode 100644 index 00000000000..3678f57e08a --- /dev/null +++ b/docs/CALLMIP_SETUP.md @@ -0,0 +1,262 @@ +# CalMIP Site Setup for ClimaLand.jl + +This directory contains the setup for running CalMIP (California Land Model Intercomparison Project) sites in ClimaLand.jl. + +## Overview + +The CalMIP project is a model intercomparison and calibration effort to improve land surface models. This setup provides a template for adding CalMIP sites to ClimaLand. + +**Current Status:** Template setup complete for DK-Sor (Denmark Sorø) test site from CalMIP Phase 1a. + +## CalMIP Resources + +- **GitHub Repository:** https://github.com/callmip-org/Phase1 +- **Website:** https://callmip-org.github.io +- **Protocol:** [CalMIP Phase 1 Protocol v1.1](https://github.com/callmip-org/Phase1/blob/main/Protocol/CalLMIP_Phase1_Protocol_v1.1.pdf) + +## File Structure + +``` +ClimaLand.jl/ +├── src/ +│ └── Artifacts.jl # Updated with callmip_data_path function +├── ext/ +│ ├── FluxnetSimulationsExt.jl # Updated to include CalMIP sites +│ └── fluxnet_simulations/ +│ └── DK-Sor.jl # Site-specific parameters +└── experiments/ + └── integrated/ + └── fluxnet/ + └── callmip_dksor.jl # Simulation runner script +``` + +**Note on Data Storage:** CalMIP forcing data files (CSV) are NOT stored in the git repository. +Like other ClimaLand sites (US-MOz, US-Ha1, etc.), these should be managed through the +ClimaArtifacts system or stored locally. See "Data Location" section below. + +## Adding a New CalMIP Site + +Follow these steps to add a new CalMIP site: + +### 1. Prepare Climate Forcing Data (CSV file) + +**Important:** CalMIP forcing data files should NOT be committed to the git repository. + +Create a CSV file with the following columns and units: + +| Column Name | Description | Units | +|-----------------|---------------------------------------|-------------| +| TIMESTAMP_START | Start of averaging period (YYYYMMDDHHMM) | - | +| TIMESTAMP_END | End of averaging period (YYYYMMDDHHMM) | - | +| TA_F | Air temperature | °C | +| VPD_F | Vapor pressure deficit | hPa | +| PA_F | Air pressure | kPa | +| P_F | Accumulated precipitation | mm | +| WS_F | Wind speed | m/s | +| LW_IN_F | Downwelling longwave radiation | W/m² | +| SW_IN_F | Downwelling shortwave radiation | W/m² | +| CO2_F_MDS | CO2 concentration | μmol/mol | + +#### Data Location Options: + +**Option 1: Environment Variable (Recommended for local testing)** +```bash +# Set the path to your CalMIP data directory +export CALLMIP_DATA_PATH="/path/to/your/callmip/data" +# Place your CSV files there: /path/to/your/callmip/data/DK-Sor.csv +``` + +**Option 2: Default Local Path** +```bash +# ClimaLand will look in: ../callmip_data/ (relative to ClimaLand.jl) +mkdir ../callmip_data +# Place your CSV files there +``` + +**Option 3: Add to ClimaArtifacts (Recommended for production)** +For permanent integration, CalMIP data should be added to the ClimaArtifacts system +(similar to how Fluxnet sites like US-MOz are stored). This requires: +1. Uploading data to Caltech's artifact storage +2. Updating the Artifacts.toml file +3. Modifying `callmip_data_path()` in src/Artifacts.jl to use `@clima_artifact` + +**Where to get CalMIP data:** +- CalMIP GitHub repository: https://github.com/callmip-org/Phase1/tree/main/Data +- Model evaluation data portal: https://modelevaluation.org +- Or obtain from Box link: https://caltech.box.com/shared/static/otrr2y0rgjct7hqhmq214nb8qjsvqj5p.gz + +Add your site to the `callmip_data_path` function in `src/Artifacts.jl`: + +```julia +function callmip_data_path(site_ID; context = nothing) + @assert site_ID ∈ ("DK-Sor", "YOUR-SITE-ID") # Add your site here + + climaland_dir = pkgdir(@__MODULE__) + data_path = joinpath(climaland_dir, "data", "callmip_sites", "$(site_ID).csv") + return data_path +end +``` + +### 3. Create Site Information File + +Create a new file in `ext/fluxnet_simulations/` named after your site (e.g., `YOUR-SITE-ID.jl`). + +Use `ext/fluxnet_simulations/DK-Sor.jl` as a template. You need to define: + +```julia +# Replace DK_Sor with your site ID (underscores instead of hyphens) +function FluxnetSimulations.get_domain_info(FT, ::Val{:YOUR_SITE_ID}; ...) +function FluxnetSimulations.get_location(FT, ::Val{:YOUR_SITE_ID}; ...) +function FluxnetSimulations.get_fluxtower_height(FT, ::Val{:YOUR_SITE_ID}; ...) +function FluxnetSimulations.get_parameters(FT, ::Val{:YOUR_SITE_ID}; ...) +``` + +**Key information needed:** +- Latitude and longitude +- Time zone offset from UTC +- Tower/measurement height +- Soil depth and properties +- Vegetation type and parameters +- Root depth and canopy height + +### 4. Update FluxnetSimulationsExt.jl + +Add your site file to the includes in `ext/FluxnetSimulationsExt.jl`: + +```julia +include("fluxnet_simulations/YOUR-SITE-ID.jl") +``` + +### 5. Create Simulation Runner + +Create a simulation file in `experiments/integrated/fluxnet/` (e.g., `callmip_yoursite.jl`). + +Use `experiments/integrated/fluxnet/callmip_dksor.jl` as a template. + +**Key modifications needed:** +- Update `site_ID = "YOUR-SITE-ID"` +- Adjust simulation dates: `start_date` and `stop_date` +- Modify output directory path if needed + +### 6. Run the Simulation + +First, ensure your CalMIP forcing data is accessible (via environment variable or local path): + +```bash +# Option 1: Set environment variable +export CALLMIP_DATA_PATH="/path/to/callmip/data" + +# Then run +cd ClimaLand.jl +julia --project=experiments experiments/integrated/fluxnet/callmip_yoursite.jl +``` + +## CalMIP-Specific Considerations + +### Output Variables + +CalMIP requires specific output variables. The current setup includes: + +**1D (surface) variables:** +- GPP (Gross Primary Productivity) +- NEE (Net Ecosystem Exchange) - calculated as ER - GPP +- ET (Evapotranspiration) +- Qle (Latent heat flux) +- Qh (Sensible heat flux) +- Rnet (Net radiation) + +**2D (profile) variables:** +- Soil water content +- Soil temperature + +Refer to the CalMIP protocol for the complete list of required outputs. + +### Calibration + +The CalMIP project focuses on model calibration. After setting up your site: + +1. Run the model with default/prior parameters +2. Compare outputs with observations (from CalMIP data repository) +3. Use calibration framework (see `experiments/calibration/`) +4. Follow CalMIP protocol for calibration procedures + +### Data Sources + +- **Forcing data:** Available from CalMIP repository or modelevaluation.org +- **Observation data:** Available from CalMIP Phase 1 data directory +- **Site metadata:** See CalMIP protocol for soil texture, PFT cover, etc. + +## Example: DK-Sor Test Site + +The DK-Sor (Denmark Sorø) site is set up as an example: + +- **Location:** 55.486°N, 11.6446°E +- **Vegetation:** Deciduous Broadleaf Forest (Beech) +- **Purpose:** CalMIP Phase 1a test calibration + +**Files:** +- Data: `data/callmip_sites/DK-Sor.csv` (example template only) +- Site config: `ext/fluxnet_simulations/DK-Sor.jl` +- Runner: `experiments/integrated/fluxnet/callmip_dksor.jl` + +## Data Format Notes + +### Forcing Data Requirements + +The CSV forcing data follows FLUXNET format conventions: +- Timestamps in format YYYYMMDDHHMM +- Half-hourly or hourly temporal resolution +- Gap-filled meteorological data preferred +- Quality flags can be included but are not required + +### Initial Conditions + +The simulation will use generic initial conditions unless you specify custom ones: +- Soil moisture: From equilibrium or prior runs +- Soil temperature: From equilibrium or prior runs +- Canopy state: LAI from MODIS or prescribed + +## Troubleshooting + +### Common Issues + +1. **Missing data file:** Ensure your CSV file is in `data/callmip_sites/` +2. **Site ID mismatch:** Use hyphens in site_ID string, underscores in Val{:SITE_ID} +3. **Forcing data format:** Verify column names match exactly (case-sensitive) +4. **Time zone issues:** Check time_offset sign (positive for east of UTC) + +### Data Processing + +If you need to convert CalMIP NetCDF data to CSV format: + +```julia +# Example conversion script (adapt as needed) +using NCDatasets +using DataFrames, CSV + +# Load CalMIP NetCDF file +ds = NCDataset("path/to/callmip_forcing.nc") + +# Extract variables and convert to DataFrame +df = DataFrame( + TIMESTAMP_START = ds["time_start"][:], + TIMESTAMP_END = ds["time_end"][:], + TA_F = ds["Tair"][:], + # ... add other variables +) + +# Save as CSV +CSV.write("data/callmip_sites/YOUR-SITE-ID.csv", df) +``` + +## References + +- CalMIP Protocol: https://github.com/callmip-org/Phase1 +- ClimaLand Documentation: https://clima.github.io/ClimaLand.jl/dev/ +- FLUXNET Data Format: https://fluxnet.org/data/fluxnet2015-dataset/ + +## Contact + +For CalMIP-specific questions, visit: https://callmip-org.github.io + +For ClimaLand integration issues, open an issue at: https://github.com/CliMA/ClimaLand.jl/issues diff --git a/docs/CALLMIP_SUMMARY.md b/docs/CALLMIP_SUMMARY.md new file mode 100644 index 00000000000..c2aead1ebde --- /dev/null +++ b/docs/CALLMIP_SUMMARY.md @@ -0,0 +1,310 @@ +# CalMIP Site Setup - Summary + +## What Was Created + +A complete template setup for running CalMIP (California Land Model Intercomparison Project) sites in ClimaLand.jl, using DK-Sor (Denmark Sorø) as an example test site. + +## Files Created/Modified + +### 1. Documentation +- **`docs/CALLMIP_SETUP.md`** - Complete setup guide with detailed instructions +- **`docs/CALLMIP_QUICKSTART.md`** - Quick reference guide for adding new sites +- **`docs/CALLMIP_SUMMARY.md`** (this file) - Overview summary + +### 2. Data Files +- **`data/callmip_sites/DK-Sor.csv`** - Example CSV template with forcing data format + - Contains 24 rows of example meteorological data + - Shows required column structure and units + - **Note:** Replace with actual CalMIP data before running real simulations + +### 3. Core Code Updates + +#### src/Artifacts.jl +- Added `callmip_data_path()` function to handle CalMIP site data paths +- Supports local data directory until CalMIP data is added to artifacts + +#### ext/fluxnet_simulations/DK-Sor.jl (NEW) +- Complete site configuration for DK-Sor test site +- Includes functions for: + - `get_domain_info()` - Soil depth and discretization + - `get_location()` - Lat/lon and time zone + - `get_fluxtower_height()` - Measurement height + - `get_parameters()` - Soil, canopy, and plant hydraulics parameters + +#### ext/FluxnetSimulationsExt.jl +- Added include for DK-Sor.jl site configuration + +#### ext/fluxnet_simulations/data_processing.jl +- Modified `read_fluxnet_data()` to handle both Fluxnet and CalMIP sites + +### 4. Simulation Runner +- **`experiments/integrated/fluxnet/callmip_dksor.jl`** (NEW) + - Complete simulation setup for DK-Sor site + - Includes soil, canopy, and coupled model configuration + - Sets up diagnostics and output + - Based on ozark_pft.jl template + +## DK-Sor Site Specifications + +### Location +- **Site:** Denmark Sorø +- **Coordinates:** 55.486°N, 11.6446°E +- **Time Zone:** UTC+1 (CET) +- **Tower Height:** 57 m + +### Vegetation +- **Type:** Deciduous Broadleaf Forest +- **Dominant Species:** Beech (Fagus sylvatica) +- **Canopy Height:** 30 m +- **Stem Height:** 25 m +- **Leaf Layer:** 5 m +- **Rooting Depth:** 1.0 m + +### Soil Domain +- **Depth:** 10 m (zmin = -10, zmax = 0) +- **Elements:** 20 +- **Porosity:** 0.50 +- **Hydraulic Conductivity:** 5×10⁻⁷ m/s + +## Required CSV Data Format + +The forcing data CSV must contain these columns: + +| Column | Description | Units | +|--------|-------------|-------| +| TIMESTAMP_START | Start of period (YYYYMMDDHHMM) | - | +| TIMESTAMP_END | End of period (YYYYMMDDHHMM) | - | +| TA_F | Air temperature | °C | +| VPD_F | Vapor pressure deficit | hPa | +| PA_F | Air pressure | kPa | +| P_F | Precipitation | mm | +| WS_F | Wind speed | m/s | +| LW_IN_F | Downwelling LW radiation | W/m² | +| SW_IN_F | Downwelling SW radiation | W/m² | +| CO2_F_MDS | CO2 concentration | μmol/mol | + +## How to Add a New CalMIP Site + +### Quick Steps: + +1. **Create forcing data CSV** + ``` + data/callmip_sites/YOUR-SITE.csv + ``` + +2. **Update Artifacts.jl** + - Add site to `callmip_data_path()` assertion + +3. **Create site configuration** + ``` + ext/fluxnet_simulations/YOUR-SITE.jl + ``` + (Copy and modify DK-Sor.jl) + +4. **Update FluxnetSimulationsExt.jl** + - Add include statement + +5. **Update data_processing.jl** + - Add site to `callmip_sites` tuple + +6. **Create simulation runner** + ``` + experiments/integrated/fluxnet/callmip_yoursite.jl + ``` + (Copy and modify callmip_dksor.jl) + +### Detailed Instructions +See `docs/CALLMIP_SETUP.md` for complete step-by-step guide. + +## Where to Get CalMIP Data + +### Forcing Data +1. **CalMIP GitHub Repository:** + - https://github.com/callmip-org/Phase1/tree/main/Data + +2. **Model Evaluation Portal:** + - https://modelevaluation.org + +3. **Box Link (example):** + - https://caltech.box.com/shared/static/otrr2y0rgjct7hqhmq214nb8qjsvqj5p.gz + +### Observation Data (for calibration) +- Available in CalMIP Phase 1 Data directory +- Includes NEE, Qle, Qh with uncertainties +- See Phase 1a-test folder for DK-Sor example + +### Site Metadata +- Refer to CalMIP Phase 1 Protocol (PDF) +- Includes soil texture, PFT cover percentages +- Available as NetCDF global attributes + +## CalMIP Output Variables + +The simulation is configured to output: + +**Surface (1D) variables:** +- `gpp` - Gross Primary Productivity +- `er` - Ecosystem Respiration +- `et` - Evapotranspiration +- `shf` - Sensible Heat Flux +- `lhf` - Latent Heat Flux +- `rn` - Net Radiation +- `swu`, `lwu` - Upwelling radiation +- `gs` - Stomatal Conductance +- `ct` - Canopy Temperature +- Others... + +**Profile (2D) variables:** +- `swc` - Soil Water Content (by depth) +- `tsoil` - Soil Temperature (by depth) +- `si` - Snow/Ice + +## Running the Example + +```bash +# Navigate to ClimaLand directory +cd ClimaLand.jl + +# Run the DK-Sor simulation +julia --project=experiments experiments/integrated/fluxnet/callmip_dksor.jl +``` + +**Note:** The example CSV contains only template data. Replace it with actual CalMIP forcing data before running real simulations. + +## Output + +Results are saved to: +``` +experiments/integrated/fluxnet/DK-Sor/callmip/out/ +``` + +Includes: +- Timeseries plots (GPP, fluxes, soil moisture, temperature) +- Diurnal cycle analysis +- Comparison with observations (if available) + +## Next Steps + +1. **Get Real Data:** + - Download actual DK-Sor forcing data from CalMIP repository + - Replace template CSV with real data + +2. **Test Run:** + - Run short simulation (1-7 days) to verify setup + - Check outputs and diagnostics + +3. **Full Simulation:** + - Extend to full simulation period (months to years) + - Generate outputs required by CalMIP protocol + +4. **Calibration:** + - Use ClimaLand calibration framework + - Follow CalMIP Phase 1 protocol for calibration targets + - See `experiments/calibration/` directory + +5. **Add More Sites:** + - Follow the same pattern for other CalMIP sites + - CalMIP Phase 1 includes multiple forest, grassland, and cropland sites + +## Important Notes + +### Site ID Naming Convention +- Use **hyphens** in file names and strings: `"DK-Sor"` +- Use **underscores** in Val types: `Val(:DK_Sor)` +- This is handled automatically by `replace_hyphen()` function + +### Data Sources Must Match +- Forcing data temporal resolution should match model timestep +- Typically half-hourly (1800s) or hourly +- Gap-fill missing data before running + +### Initial Conditions +- Generic initial conditions are used by default +- For better results, spin up the model for 1-2 years +- Or use initial conditions from previous runs + +### Calibration Targets +- CalMIP specifies which fluxes to calibrate against +- Typically: NEE, Qle, Qh with uncertainties provided +- See CalMIP protocol for full list + +## References + +### CalMIP Project +- **Website:** https://callmip-org.github.io +- **GitHub:** https://github.com/callmip-org/Phase1 +- **Protocol:** CalMIP Phase 1 Protocol v1.1 (PDF in repository) + +### ClimaLand +- **Documentation:** https://clima.github.io/ClimaLand.jl/dev/ +- **GitHub:** https://github.com/CliMA/ClimaLand.jl +- **Issues:** https://github.com/CliMA/ClimaLand.jl/issues + +### FLUXNET +- **Data Format:** https://fluxnet.org/data/fluxnet2015-dataset/ +- Used as basis for CalMIP forcing data format + +## Troubleshooting + +### Common Issues + +**"Site not found" error:** +- Ensure site is added to all necessary files (see checklist above) +- Check spelling consistency (DK-Sor vs DK_Sor) + +**"Cannot find CSV file":** +- Verify file path: `data/callmip_sites/SITE-ID.csv` +- Check file permissions + +**"Method not found for Val":** +- Site configuration file not included in FluxnetSimulationsExt.jl +- Val type uses underscore, not hyphen + +**"Data format error":** +- Verify CSV has exactly the required columns +- Check for missing headers +- Ensure numeric data is not quoted + +### Getting Help + +- **CalMIP questions:** Visit https://callmip-org.github.io +- **ClimaLand questions:** Open issue at ClimaLand.jl GitHub +- **Documentation:** Read `docs/CALLMIP_SETUP.md` for detailed guide + +## File Checklist + +When adding a new site, ensure you've created/modified: + +- [ ] CSV forcing data file +- [ ] Site configuration (.jl file) +- [ ] Artifacts.jl (add site to assertion) +- [ ] FluxnetSimulationsExt.jl (include site file) +- [ ] data_processing.jl (add to callmip_sites tuple) +- [ ] Simulation runner script +- [ ] Test with short simulation +- [ ] Verify outputs + +## Example Site Comparison + +| Aspect | DK-Sor (CalMIP) | US-MOz (Fluxnet) | +|--------|-----------------|------------------| +| Location | Denmark | Missouri, USA | +| Vegetation | Beech Forest | Oak Forest | +| Canopy Height | 30 m | 18.5 m | +| Time Zone | UTC+1 | UTC-6 | +| Data Source | CalMIP | AmeriFlux | +| Purpose | Calibration Test | Model Validation | + +Both use the same underlying ClimaLand model structure and follow similar setup procedures. + +## Version Information + +- **ClimaLand.jl:** Compatible with current main branch +- **CalMIP Protocol:** Phase 1 Protocol v1.1 +- **Date Created:** February 2026 +- **Status:** Template setup complete, awaiting real forcing data + +--- + +**For detailed instructions, see:** `docs/CALLMIP_SETUP.md` +**For quick reference, see:** `docs/CALLMIP_QUICKSTART.md` diff --git a/experiments/integrated/fluxnet/CALLMIP_README.md b/experiments/integrated/fluxnet/CALLMIP_README.md new file mode 100644 index 00000000000..7630ff02dd3 --- /dev/null +++ b/experiments/integrated/fluxnet/CALLMIP_README.md @@ -0,0 +1,219 @@ +# CalMIP DK-Sor Site Simulation Guide + +This directory contains the setup for running CalMIP (Calibration Land Model Intercomparison Project) simulations for the DK-Sor (Denmark Sorø) flux tower site in ClimaLand.jl. + +## Files Created + +### 1. Site Configuration +- **Location**: `ext/fluxnet_simulations/DK-Sor.jl` +- Contains site-specific parameters: + - Location: 55.486°N, 11.6446°E, UTC+1 + - Vegetation: Deciduous Broadleaf Forest (Beech) + - Soil parameters: porosity, hydraulic conductivity, van Genuchten parameters + - Canopy parameters: height (30m), Vcmax25, stomatal conductance + +### 2. Simulation Runner +- **File**: `experiments/integrated/fluxnet/callmip_dksor.jl` +- Runs full year simulation (2008) +- Timestep: 450 seconds (7.5 minutes) +- Saves 16 diagnostic variables at half-hourly resolution + +### 3. Forcing Data +- **Location**: `/net/sampo/data1/renatob/callmip_forcing/DK-Sor.csv` +- **Source**: Converted from `DK-Sor_1997-2014_FLUXNET2015_Met.nc` +- **Variables**: Air temperature, VPD, pressure, precipitation, wind, radiation, CO2 +- **Period**: 1997-2015 (315,552 half-hourly timesteps) +- **Conversion Script**: `convert_netcdf_fast.py` (Python) + +### 4. Observation Data +- **Location**: `/net/sampo/data1/renatob/DK-Sor_daily_aggregated_1997-2013_FLUXNET2015_Flux.nc` +- **Variables**: LE (latent heat), H (sensible heat), NEE (net ecosystem exchange) +- **Usage**: For model validation and comparison + +### 5. Analysis Tools + +#### Comparison Script +- **File**: `compare_callmip_observations.py` +- **Purpose**: Compare simulation outputs with flux observations +- **Features**: + - Reads HDF5 simulation outputs + - Reads NetCDF flux observations + - Calculates statistics (RMSE, bias, correlation) + - Creates timeseries and scatter plots + +#### Monitoring Script +- **File**: `monitor_callmip.sh` +- **Purpose**: Monitor simulation progress +- **Usage**: `./monitor_callmip.sh` + +## Running the Simulation + +### Quick Start +```bash +cd $HOME/ClimaLand.jl +julia --project=. experiments/integrated/fluxnet/callmip_dksor.jl +``` + +### Background Run (for long simulations) +```bash +cd $HOME/ClimaLand.jl +nohup julia --project=. experiments/integrated/fluxnet/callmip_dksor.jl > callmip_run.log 2>&1 & +``` + +### Monitor Progress +```bash +./experiments/integrated/fluxnet/monitor_callmip.sh +# or +tail -f callmip_run.log +``` + +## Output Structure + +### Simulation Outputs +- **Directory**: `experiments/integrated/fluxnet/DK-Sor/callmip/out/` +- **Format**: HDF5 files (one per diagnostic variable) +- **Frequency**: Half-hourly (30-minute intervals) + +### Diagnostic Variables (16 total) +Key outputs include: +- `lhf.hdf5` - Latent heat flux (W/m²) +- `shf.hdf5` - Sensible heat flux (W/m²) +- `gpp.hdf5` - Gross primary production (μmol/m²/s) +- `sw.hdf5` - Soil water content (m³/m³) +- `st.hdf5` - Soil temperature (K) +- Additional canopy and soil variables + +## Comparing with Observations + +Once the simulation completes, compare with observations: + +```bash +python3 experiments/integrated/fluxnet/compare_callmip_observations.py \ + --flux-file /net/sampo/data1/renatob/DK-Sor_daily_aggregated_1997-2013_FLUXNET2015_Flux.nc \ + --output-dir experiments/integrated/fluxnet/DK-Sor/callmip/out/ \ + --start-date 2008-01-01 \ + --end-date 2009-01-01 \ + --plot-dir experiments/integrated/fluxnet/DK-Sor/callmip/plots/ +``` + +This will generate: +- Timeseries comparison plots for LE, H, NEE +- Scatter plots with statistics (RMSE, bias, R) +- Console output with quantitative metrics + +## Data Processing Pipeline + +### NetCDF → CSV Conversion +The forcing data was converted using: +```bash +python3 experiments/integrated/fluxnet/convert_netcdf_fast.py \ + /net/sampo/data1/renatob/DK-Sor_1997-2014_FLUXNET2015_Met.nc \ + /net/sampo/data1/renatob/callmip_forcing/DK-Sor.csv +``` + +### Data Flow +``` +NetCDF Forcing → CSV → ClimaLand Simulation → HDF5 Outputs + ↓ +NetCDF Observations ← ← ← ← ← ← Comparison Script +``` + +## Key Modifications to ClimaLand + +1. **Artifacts.jl** + - Added `callmip_data_path()` function + - Points to `/net/sampo/data1/renatob/callmip_forcing/` + +2. **data_processing.jl** + - Updated `read_fluxnet_data()` to recognize CalMIP sites + - Checks for "DK-Sor" and routes to CalMIP data path + +3. **initial_conditions.jl** + - Updated `set_fluxnet_ic!()` to handle CalMIP sites + - Routes CalMIP sites to appropriate data path + +## Simulation Details + +### Model Components +- **Soil**: Energy-Hydrology model with biogeochemistry + - 20 layers, exponentially stretched + - Depth: 0 to -10m + - van Genuchten hydraulic model + +- **Canopy**: Coupled photosynthesis and plant hydraulics + - Two-stream radiative transfer + - Farquhar photosynthesis model + - Medlyn stomatal conductance + - Plant hydraulics with Weibull vulnerability curve + +- **CO2**: Soil CO2 production and transport + +### Timestepping +- Integration: IMEX (Implicit-Explicit) scheme +- Timestep: 450 seconds (7.5 minutes) +- Adaptive stepping with Newton solver + +### Boundary Conditions +- Top: Atmospheric forcing from CSV data +- Bottom: Zero flux (deep drainage allowed through runoff) + +## Troubleshooting + +### Common Issues + +1. **Package not found errors** + - Solution: Use `--project=.` flag (not `--project=experiments`) + - Ensures proper package environment + +2. **Data path errors** + - Check that CSV file exists at: `/net/sampo/data1/renatob/callmip_forcing/DK-Sor.csv` + - Verify file permissions + +3. **Slow compilation** + - First run takes 2-5 minutes to compile + - Subsequent runs are much faster + - Use `--compile=min` to reduce compilation time + +4. **Memory issues** + - Year-long simulation uses ~1.5-2GB RAM + - Reduce diagnostic frequency if needed + +### Checking Simulation Status +```bash +# Check if running +ps aux | grep "julia.*callmip_dksor" + +# Check progress +tail -f callmip_run.log + +# Check output files +ls -lh experiments/integrated/fluxnet/DK-Sor/callmip/out/ +``` + +## Performance Expectations + +- **Compilation time**: 2-5 minutes (first run) +- **Simulation time** (1 year, half-hourly output): + - Expected: 5-15 minutes on typical workstation + - Varies with CPU speed and available cores +- **Output size**: ~100-500 MB total (all diagnostics) + +## Next Steps + +1. ✅ Run full year simulation (2008) +2. ⏳ Wait for completion (~10-15 min) +3. 📊 Compare with observations +4. 📈 Generate validation plots +5. 📝 Calculate performance metrics +6. 🔄 Extend to multi-year runs if needed + +## References + +- CalMIP Protocol: https://github.com/callmip-org/Phase1 +- FLUXNET2015: https://fluxnet.org/ +- DK-Sor Site: Deciduous beech forest in Denmark +- ClimaLand.jl: https://github.com/CliMA/ClimaLand.jl + +## Contact + +For issues or questions about this CalMIP setup, see the ClimaLand.jl repository or CalMIP protocol documentation. diff --git a/experiments/integrated/fluxnet/callmip_dksor.jl b/experiments/integrated/fluxnet/callmip_dksor.jl new file mode 100644 index 00000000000..4cfa1173718 --- /dev/null +++ b/experiments/integrated/fluxnet/callmip_dksor.jl @@ -0,0 +1,337 @@ +""" +This experiment runs the DK-Sor (Denmark Sorø) CalMIP Phase 1a test site. + +The DK-Sor site is a deciduous broadleaf forest in Denmark, dominated by beech trees. +This is the test calibration site for CalMIP Phase 1a. + +For more information about the CalMIP project and protocols: +https://github.com/callmip-org/Phase1 +https://callmip-org.github.io + +Citation: CalMIP Phase 1 Protocol v1.1 +""" + +import ClimaLand +import SciMLBase +using ClimaCore +import ClimaComms +import ClimaParams as CP +using Dates +using Insolation + +using ClimaLand +using ClimaLand.Domains: Column +using ClimaLand.Soil +using ClimaLand.Soil.Biogeochemistry +using ClimaLand.Canopy +using ClimaLand.Canopy.PlantHydraulics +import ClimaLand.Simulations: LandSimulation, solve! +import ClimaLand +import ClimaLand.Parameters as LP +using ClimaDiagnostics +using ClimaUtilities + +using DelimitedFiles +import ClimaLand.FluxnetSimulations as FluxnetSimulations +# using CairoMakie, ClimaAnalysis, GeoMakie, Printf, StatsBase +# import ClimaLand.LandSimVis as LandSimVis + +const FT = Float64 +toml_dict = LP.create_toml_dict(FT) +climaland_dir = pkgdir(ClimaLand) + +# Site identification +site_ID = "DK-Sor" +site_ID_val = FluxnetSimulations.replace_hyphen(site_ID) + +# Get the default values for this site's domain, location, and parameters +(; dz_tuple, nelements, zmin, zmax) = + FluxnetSimulations.get_domain_info(FT, Val(site_ID_val)) +(; time_offset, lat, long) = + FluxnetSimulations.get_location(FT, Val(site_ID_val)) +(; atmos_h) = FluxnetSimulations.get_fluxtower_height(FT, Val(site_ID_val)) + +# Get site parameters +(; + soil_ν, + soil_K_sat, + soil_S_s, + soil_vg_n, + soil_vg_α, + θ_r, + ν_ss_quartz, + ν_ss_om, + ν_ss_gravel, + z_0m_soil, + z_0b_soil, + soil_ϵ, + soil_α_PAR, + soil_α_NIR, + Ω, + χl, + α_PAR_leaf, + λ_γ_PAR, + τ_PAR_leaf, + α_NIR_leaf, + τ_NIR_leaf, + ϵ_canopy, + ac_canopy, + g1, + Drel, + g0, + Vcmax25, + SAI, + f_root_to_shoot, + K_sat_plant, + ψ63, + Weibull_param, + a, + conductivity_model, + retention_model, + plant_ν, + plant_S_s, + rooting_depth, + n_stem, + n_leaf, + h_leaf, + h_stem, + h_canopy, +) = FluxnetSimulations.get_parameters(FT, Val(site_ID_val)) + +compartment_midpoints = + n_stem > 0 ? [h_stem / 2, h_stem + h_leaf / 2] : [h_leaf / 2] +compartment_surfaces = n_stem > 0 ? [zmax, h_stem, h_canopy] : [zmax, h_leaf] + +# Construct the ClimaLand domain to run the simulation on +land_domain = Column(; + zlim = (zmin, zmax), + nelements = nelements, + dz_tuple = dz_tuple, + longlat = (long, lat), +) +canopy_domain = ClimaLand.Domains.obtain_surface_domain(land_domain) +prognostic_land_components = (:canopy, :soil, :soilco2) + +# Set up the timestepping information for the simulation +dt = Float64(450) # 7.5 minutes +# Define simulation period - short 1-day test +start_date = DateTime(2008, 1, 1) +stop_date = DateTime(2009, 1, 1) # Full year 2008 + +# This reads in the data from the CalMIP site and creates +# the atmospheric and radiative driver structs for the model. +# CalMIP-format data for this site are handled via the shared +# Fluxnet/CalMIP data-reading utilities used by `prescribed_forcing_fluxnet`. +(; atmos, radiation) = FluxnetSimulations.prescribed_forcing_fluxnet( + site_ID, + lat, + long, + time_offset, + atmos_h, + start_date, + toml_dict, + FT; + split_precip = :callmip, +) + +# Set up soil model +soil_domain = land_domain +soil_albedo = Soil.ConstantTwoBandSoilAlbedo{FT}(; + PAR_albedo = soil_α_PAR, + NIR_albedo = soil_α_NIR, +) + +runoff = ClimaLand.Soil.Runoff.SurfaceRunoff() +retention_parameters = (; + ν = soil_ν, + θ_r, + K_sat = soil_K_sat, + hydrology_cm = vanGenuchten{FT}(; α = soil_vg_α, n = soil_vg_n), +) +composition_parameters = (; ν_ss_om, ν_ss_quartz, ν_ss_gravel) +soil_forcing = (; atmos, radiation) +soil = Soil.EnergyHydrology{FT}( + soil_domain, + soil_forcing, + toml_dict; + prognostic_land_components, + additional_sources = (ClimaLand.RootExtraction{FT}(),), + albedo = soil_albedo, + runoff, + retention_parameters, + composition_parameters, + S_s = soil_S_s, + z_0m = z_0m_soil, + z_0b = z_0b_soil, + emissivity = soil_ϵ, +) + +# Soil microbes model +co2_prognostic_soil = Soil.Biogeochemistry.PrognosticMet(soil.parameters) +drivers = Soil.Biogeochemistry.SoilDrivers(co2_prognostic_soil, atmos) +soilco2 = Soil.Biogeochemistry.SoilCO2Model{FT}(soil_domain, drivers, toml_dict) + +# Set up canopy radiative transfer +radiation_parameters = (; + Ω, + G_Function = CLMGFunction(χl), + α_PAR_leaf, + τ_PAR_leaf, + α_NIR_leaf, + τ_NIR_leaf, +) +radiative_transfer = Canopy.TwoStreamModel{FT}( + canopy_domain, + toml_dict; + radiation_parameters, + ϵ_canopy, +) + +# Set up stomatal conductance +conductance = Canopy.MedlynConductanceModel{FT}(canopy_domain, toml_dict; g1) + +# Set up photosynthesis +photosynthesis_parameters = (; is_c3 = FT(1), Vcmax25) +photosynthesis = + FarquharModel{FT}(canopy_domain, toml_dict; photosynthesis_parameters) + +# Set up plant hydraulics +surface_space = land_domain.space.surface + +# Use prescribed LAI from MODIS data or constant LAI for testing +LAI = + ClimaLand.Canopy.prescribed_lai_modis(surface_space, start_date, stop_date) +# Get the maximum LAI at this site over the first year of the simulation +maxLAI = FluxnetSimulations.get_maxLAI_at_site(start_date, lat, long) +RAI = maxLAI * f_root_to_shoot + +hydraulics = Canopy.PlantHydraulicsModel{FT}( + canopy_domain, + toml_dict; + n_stem, + n_leaf, + h_stem, + h_leaf, + ν = plant_ν, + S_s = plant_S_s, + conductivity_model, + retention_model, +) + +height = h_stem + h_leaf +biomass = + Canopy.PrescribedBiomassModel{FT}(; LAI, SAI, RAI, rooting_depth, height) + +# Set up canopy energy model +energy = Canopy.BigLeafEnergyModel{FT}(toml_dict; ac_canopy) + +ground = ClimaLand.PrognosticGroundConditions{FT}() +canopy_forcing = (; atmos, radiation, ground) + +# Construct the canopy model +canopy = Canopy.CanopyModel{FT}( + canopy_domain, + canopy_forcing, + LAI, + toml_dict; + prognostic_land_components, + radiative_transfer, + photosynthesis, + conductance, + hydraulics, + energy, + biomass, +) + +# Integrated land model +land = SoilCanopyModel{FT}(soilco2, soil, canopy) + +# Set initial conditions +set_ic! = FluxnetSimulations.make_set_fluxnet_initial_conditions( + site_ID, + start_date, + time_offset, + land, +) + +# Set up diagnostics +# Define output directory for diagnostics +outdir = joinpath( + pkgdir(ClimaLand), + "experiments/integrated/fluxnet/$(site_ID)/callmip/out", +) +mkpath(outdir) +println("Creating NetCDF output writer for directory: ", outdir) + +# Use NetCDFWriter to save diagnostics to NetCDF files +output_writer = ClimaDiagnostics.Writers.NetCDFWriter( + soil_domain.space.subsurface, + outdir; + start_date, +) + +# CalMIP requires specific output variables - ensure these match protocol requirements +short_names_1D = [ + "sif", # Solar-induced fluorescence + "ra", # Autotrophic respiration + "gs", # Stomatal conductance + "gpp", # Gross primary productivity + "ct", # Canopy temperature + "swu", # Upwelling shortwave radiation + "lwu", # Upwelling longwave radiation + "er", # Ecosystem respiration + "et", # Evapotranspiration + "msf", # Moisture surface flux + "shf", # Sensible heat flux + "lhf", # Latent heat flux + "rn", # Net radiation +] +short_names_2D = [ + "swc", # Soil water content + "tsoil", # Soil temperature + "si", # Snow/ice +] +output_vars = [short_names_1D..., short_names_2D...] + +diags = ClimaLand.default_diagnostics( + land, + start_date; + output_writer = output_writer, + output_vars, + reduction_period = :halfhourly, +) + +# Run the simulation +simulation = LandSimulation( + start_date, + stop_date, + dt, + land; + set_ic! = set_ic!, + updateat = Second(dt), + diagnostics = diags, +) +solve!(simulation) + +# Post-processing +println("\nSimulation completed successfully!") +println("Start date: ", start_date) +println("Stop date: ", stop_date) +println("Number of diagnostics: ", length(diags)) + +# Close the output writers to flush diagnostics to files +ClimaLand.Diagnostics.close_output_writers(diags) +println("Output files written to: ", outdir) + +# List output files +println("Output files:") +for file in readdir(outdir; join = false) + filepath = joinpath(outdir, file) + if isfile(filepath) + filesize_mb = round(stat(filepath).size / 1024^2, digits = 2) + println(" $file ($(filesize_mb) MB)") + end +end + +# TODO: Add visualization when CairoMakie is available +# TODO: Add comparison with observation data from Flux NetCDF file diff --git a/experiments/integrated/fluxnet/callmip_dksor_default.jl b/experiments/integrated/fluxnet/callmip_dksor_default.jl new file mode 100644 index 00000000000..391959a7049 --- /dev/null +++ b/experiments/integrated/fluxnet/callmip_dksor_default.jl @@ -0,0 +1,176 @@ +# ── DK-Sor CalMIP Phase 1a: Default LandModel Configuration ────────────────── +# +# This script runs the DK-Sor (Denmark Sorø) CalMIP Phase 1a site using the +# high-level LandModel constructor, which auto-configures all sub-models +# (soil, canopy, snow, soilCO2) from toml/spatial defaults. +# +# Default sub-models: +# - Photosynthesis: FarquharModel +# - Conductance: MedlynConductanceModel +# - Radiative transfer: TwoStreamModel +# - Soil moisture stress: TuzetMoistureStressModel +# - Plant hydraulics: PlantHydraulicsModel +# - Energy: BigLeafEnergyModel +# - Snow: SnowModel (with default albedo) +# +# For more information about the CalMIP project: +# https://github.com/callmip-org/Phase1 +# https://callmip-org.github.io + +# ── 1. Imports ──────────────────────────────────────────────────────────────── +import ClimaLand +import SciMLBase +using ClimaCore +import ClimaComms +import ClimaParams as CP +using Dates +using Insolation + +using ClimaLand +using ClimaLand: LandModel +using ClimaLand.Domains: Column +using ClimaLand.Soil +using ClimaLand.Soil.Biogeochemistry +using ClimaLand.Canopy +using ClimaLand.Canopy.PlantHydraulics +using ClimaLand.Snow +import ClimaLand.Simulations: LandSimulation, solve! +import ClimaLand.Parameters as LP +import ClimaLand.FluxnetSimulations as FluxnetSimulations +using ClimaDiagnostics +using ClimaUtilities + +# ── 2. Configuration ────────────────────────────────────────────────────────── +const FT = Float64 +site_ID = "DK-Sor" +site_ID_val = FluxnetSimulations.replace_hyphen(site_ID) +climaland_dir = pkgdir(ClimaLand) +toml_dict = LP.create_toml_dict(FT) + +# Simulation parameters +dt = Float64(450) # 7.5 minutes +# CalMIP Phase 1a simulation period (must be within MODIS LAI coverage: 2000-2020) +start_date = DateTime(2008, 1, 1) +stop_date = DateTime(2009, 1, 1) # Full year 2008 + +# ── 3. Site metadata ────────────────────────────────────────────────────────── +(; dz_tuple, nelements, zmin, zmax) = + FluxnetSimulations.get_domain_info(FT, Val(site_ID_val)) +(; time_offset, lat, long) = + FluxnetSimulations.get_location(FT, Val(site_ID_val)) +(; atmos_h) = FluxnetSimulations.get_fluxtower_height(FT, Val(site_ID_val)) + +# ── 4. Domain ───────────────────────────────────────────────────────────────── +land_domain = Column(; + zlim = (zmin, zmax), + nelements = nelements, + dz_tuple = dz_tuple, + longlat = (long, lat), +) +surface_space = land_domain.space.surface + +# ── 5. Atmospheric & radiation forcing ──────────────────────────────────────── +(; atmos, radiation) = FluxnetSimulations.prescribed_forcing_fluxnet( + site_ID, + lat, + long, + time_offset, + atmos_h, + start_date, + toml_dict, + FT; + split_precip = :callmip, +) + +# ── 6. LAI from MODIS ───────────────────────────────────────────────────────── +LAI = + ClimaLand.Canopy.prescribed_lai_modis(surface_space, start_date, stop_date) + +# ── 7. Build LandModel (all defaults) ──────────────────────────────────────── +prognostic_land_components = (:canopy, :snow, :soil, :soilco2) +forcing = (; atmos, radiation) + +land = LandModel{FT}( + forcing, + LAI, + toml_dict, + land_domain, + dt; + prognostic_land_components, +) + +# ── 8. Initial conditions ──────────────────────────────────────────────────── +set_ic! = FluxnetSimulations.make_set_fluxnet_initial_conditions( + site_ID, + start_date, + time_offset, + land, +) + +# ── 9. Diagnostics ─────────────────────────────────────────────────────────── +outdir = joinpath( + pkgdir(ClimaLand), + "experiments/integrated/fluxnet/$(site_ID)/callmip_default/out", +) +mkpath(outdir) +println("Output directory: ", outdir) + +output_writer = ClimaDiagnostics.Writers.NetCDFWriter( + land_domain.space.subsurface, + outdir; + start_date, +) + +short_names_1D = [ + "sif", + "ra", + "gs", + "gpp", + "ct", + "swu", + "lwu", + "er", + "et", + "msf", + "shf", + "lhf", + "rn", +] +short_names_2D = ["swc", "tsoil", "si", "sco2", "soc", "so2"] +output_vars = [short_names_1D..., short_names_2D..., "swe"] + +diags = ClimaLand.default_diagnostics( + land, + start_date; + output_writer = output_writer, + output_vars, + reduction_period = :halfhourly, +) + +# ── 10. Run simulation ─────────────────────────────────────────────────────── +simulation = LandSimulation( + start_date, + stop_date, + dt, + land; + set_ic!, + updateat = Second(dt), + diagnostics = diags, +) +solve!(simulation) + +# ── 11. Post-processing ────────────────────────────────────────────────────── +println("\nSimulation completed successfully!") +println("Start date: ", start_date) +println("Stop date: ", stop_date) + +ClimaLand.Diagnostics.close_output_writers(diags) +println("Output files written to: ", outdir) + +for file in readdir(outdir; join = false) + filepath = joinpath(outdir, file) + if isfile(filepath) + filesize_mb = round(stat(filepath).size / 1024^2, digits = 2) + println(" $file ($(filesize_mb) MB)") + end +end diff --git a/experiments/integrated/fluxnet/callmip_dksor_pmodel.jl b/experiments/integrated/fluxnet/callmip_dksor_pmodel.jl new file mode 100644 index 00000000000..081166627eb --- /dev/null +++ b/experiments/integrated/fluxnet/callmip_dksor_pmodel.jl @@ -0,0 +1,205 @@ +# ── DK-Sor CalMIP Phase 1a: PModel Configuration ───────────────────────────── +# +# This script runs the DK-Sor (Denmark Sorø) CalMIP Phase 1a site using the +# high-level LandModel constructor with PModel photosynthesis overrides. +# +# Overridden sub-models: +# - Photosynthesis: PModel (optimality-based) +# - Conductance: PModelConductance (coupled with PModel) +# - Soil moisture stress: PiecewiseMoistureStressModel (θ-based, not ψ-based) +# +# Default sub-models (from LandModel constructor): +# - Radiative transfer: TwoStreamModel +# - Plant hydraulics: PlantHydraulicsModel +# - Energy: BigLeafEnergyModel +# - Snow: SnowModel (with default albedo) +# - Soil: EnergyHydrology (spatial defaults) +# - SoilCO2: SoilCO2Model +# +# For more information about the CalMIP project: +# https://github.com/callmip-org/Phase1 +# https://callmip-org.github.io + +# ── 1. Imports ──────────────────────────────────────────────────────────────── +import ClimaLand +import SciMLBase +using ClimaCore +import ClimaComms +import ClimaParams as CP +using Dates +using Insolation + +using ClimaLand +using ClimaLand: LandModel +using ClimaLand.Domains: Column +using ClimaLand.Soil +using ClimaLand.Soil.Biogeochemistry +using ClimaLand.Canopy +using ClimaLand.Canopy.PlantHydraulics +using ClimaLand.Snow +import ClimaLand.Simulations: LandSimulation, solve! +import ClimaLand.Parameters as LP +import ClimaLand.FluxnetSimulations as FluxnetSimulations +using ClimaDiagnostics +using ClimaUtilities + +# ── 2. Configuration ────────────────────────────────────────────────────────── +const FT = Float64 +site_ID = "DK-Sor" +site_ID_val = FluxnetSimulations.replace_hyphen(site_ID) +climaland_dir = pkgdir(ClimaLand) +toml_dict = LP.create_toml_dict(FT) + +# Simulation parameters +dt = Float64(450) # 7.5 minutes +# CalMIP Phase 1a simulation period (must be within MODIS LAI coverage: 2000-2020) +start_date = DateTime(2008, 1, 1) +stop_date = DateTime(2009, 1, 1) # Full year 2008 + +# ── 3. Site metadata ────────────────────────────────────────────────────────── +(; dz_tuple, nelements, zmin, zmax) = + FluxnetSimulations.get_domain_info(FT, Val(site_ID_val)) +(; time_offset, lat, long) = + FluxnetSimulations.get_location(FT, Val(site_ID_val)) +(; atmos_h) = FluxnetSimulations.get_fluxtower_height(FT, Val(site_ID_val)) + +# ── 4. Domain ───────────────────────────────────────────────────────────────── +land_domain = Column(; + zlim = (zmin, zmax), + nelements = nelements, + dz_tuple = dz_tuple, + longlat = (long, lat), +) +surface_space = land_domain.space.surface +canopy_domain = ClimaLand.Domains.obtain_surface_domain(land_domain) + +# ── 5. Atmospheric & radiation forcing ──────────────────────────────────────── +(; atmos, radiation) = FluxnetSimulations.prescribed_forcing_fluxnet( + site_ID, + lat, + long, + time_offset, + atmos_h, + start_date, + toml_dict, + FT; + split_precip = :callmip, +) + +# ── 6. LAI from MODIS ───────────────────────────────────────────────────────── +LAI = + ClimaLand.Canopy.prescribed_lai_modis(surface_space, start_date, stop_date) + +# ── 7. Custom canopy components (PModel overrides) ──────────────────────────── +prognostic_land_components = (:canopy, :snow, :soil, :soilco2) +forcing = (; atmos, radiation) + +# PModel photosynthesis (optimality-based, requires PModelConductance) +photosynthesis = PModel{FT}(canopy_domain, toml_dict) +conductance = PModelConductance{FT}(toml_dict) + +# Soil moisture stress based on soil moisture (θ) rather than leaf water potential (ψ) +soil_moisture_stress = + ClimaLand.Canopy.PiecewiseMoistureStressModel{FT}(land_domain, toml_dict) + +ground = ClimaLand.PrognosticGroundConditions{FT}() +canopy_forcing = (; atmos, radiation, ground) + +# Construct canopy with PModel overrides; other components use defaults +canopy = ClimaLand.Canopy.CanopyModel{FT}( + canopy_domain, + canopy_forcing, + LAI, + toml_dict; + prognostic_land_components, + photosynthesis, + conductance, + soil_moisture_stress, +) + +# ── 8. Build LandModel with custom canopy ───────────────────────────────────── +land = LandModel{FT}( + forcing, + LAI, + toml_dict, + land_domain, + dt; + prognostic_land_components, + canopy, +) + +# ── 9. Initial conditions ──────────────────────────────────────────────────── +set_ic! = FluxnetSimulations.make_set_fluxnet_initial_conditions( + site_ID, + start_date, + time_offset, + land, +) + +# ── 10. Diagnostics ────────────────────────────────────────────────────────── +outdir = joinpath( + pkgdir(ClimaLand), + "experiments/integrated/fluxnet/$(site_ID)/callmip_pmodel/out", +) +mkpath(outdir) +println("Output directory: ", outdir) + +output_writer = ClimaDiagnostics.Writers.NetCDFWriter( + land_domain.space.subsurface, + outdir; + start_date, +) + +short_names_1D = [ + "sif", + "ra", + "gs", + "gpp", + "ct", + "swu", + "lwu", + "er", + "et", + "msf", + "shf", + "lhf", + "rn", +] +short_names_2D = ["swc", "tsoil", "si", "sco2", "soc", "so2"] +output_vars = [short_names_1D..., short_names_2D..., "swe"] + +diags = ClimaLand.default_diagnostics( + land, + start_date; + output_writer = output_writer, + output_vars, + reduction_period = :halfhourly, +) + +# ── 11. Run simulation ─────────────────────────────────────────────────────── +simulation = LandSimulation( + start_date, + stop_date, + dt, + land; + set_ic!, + updateat = Second(dt), + diagnostics = diags, +) +solve!(simulation) + +# ── 12. Post-processing ────────────────────────────────────────────────────── +println("\nSimulation completed successfully!") +println("Start date: ", start_date) +println("Stop date: ", stop_date) + +ClimaLand.Diagnostics.close_output_writers(diags) +println("Output files written to: ", outdir) + +for file in readdir(outdir; join = false) + filepath = joinpath(outdir, file) + if isfile(filepath) + filesize_mb = round(stat(filepath).size / 1024^2, digits = 2) + println(" $file ($(filesize_mb) MB)") + end +end diff --git a/ext/FluxnetSimulationsExt.jl b/ext/FluxnetSimulationsExt.jl index 2ec50608469..db2071e9cfb 100644 --- a/ext/FluxnetSimulationsExt.jl +++ b/ext/FluxnetSimulationsExt.jl @@ -27,6 +27,7 @@ include("fluxnet_simulations/US-MOz.jl") include("fluxnet_simulations/US-Ha1.jl") include("fluxnet_simulations/US-NR1.jl") include("fluxnet_simulations/US-Var.jl") +include("fluxnet_simulations/DK-Sor.jl") # CalMIP Phase 1a test site # Include the data processing, forcing, and initial conditions utilities. include("fluxnet_simulations/data_processing.jl") diff --git a/ext/fluxnet_simulations/DK-Sor.jl b/ext/fluxnet_simulations/DK-Sor.jl new file mode 100644 index 00000000000..49ff0569903 --- /dev/null +++ b/ext/fluxnet_simulations/DK-Sor.jl @@ -0,0 +1,196 @@ +#= +This file contains domain specifications, parameters, and other site-level information for running +ClimaLand at the DK-Sor (Denmark Sorø) CalMIP site. + +Site information: +- Location: Sorø, Denmark (55.486°N, 11.6446°E) +- Vegetation type: Deciduous Broadleaf Forest (Beech - Fagus sylvatica) +- CalMIP Phase 1a test calibration site +- Reference: https://github.com/callmip-org/Phase1 +=# + +""" + FluxnetSimulations.get_domain_info(FT, ::Val{:DK_Sor}; dz_bottom = FT(1.5), dz_top = FT(0.1), + nelements = 20, zmin = FT(-10), zmax = FT(0)) + +Gets and returns primary domain information for the DK-Sor (Denmark Sorø) CalMIP site. +Default parameters are provided and can be overriden using keyword arguments. +""" +function FluxnetSimulations.get_domain_info( + FT, + ::Val{:DK_Sor}; + dz_bottom = FT(1.5), + dz_top = FT(0.1), + nelements = 20, + zmin = FT(-10), + zmax = FT(0), +) + dz_tuple = (dz_bottom, dz_top) + return (; dz_tuple, nelements, zmin, zmax) +end + +""" + get_location(FT, ::Val{:DK_Sor}; kwargs) + +Returns geographical information for DK-Sor (Denmark Sorø) CalMIP site. +The values are provided as defaults, and can be overwritten by passing the +corresponding keyword arguments to this function. + +The `time_offset` is the difference from UTC in hours +and excludes daylight savings time, following Fluxnet convention. +For this site, Denmark is in the Central European Time zone (CET), which is UTC+1. +""" +function FluxnetSimulations.get_location( + FT, + ::Val{:DK_Sor}; + time_offset = 1, + lat = FT(55.486), + long = FT(11.6446), +) + return (; time_offset, lat, long) +end + +""" + get_fluxtower_height(FT, ::Val{:DK_Sor}; kwargs...) + +Returns atmosphere height for DK-Sor (Denmark Sorø) CalMIP site. +The values are provided as defaults, and can be overwritten by passing the +corresponding keyword arguments to this function. +""" +function FluxnetSimulations.get_fluxtower_height( + FT, + ::Val{:DK_Sor}; + atmos_h = FT(57), +) + return (; atmos_h,) +end + +""" + get_parameters(FT, ::Val{:DK_Sor}; kwargs...) + +Gets parameters for the CalMIP site DK-Sor (Denmark Sorø), which is a Deciduous Broadleaf Forest +dominated by beech trees (Fagus sylvatica), and returns them as a Named Tuple. +The values are provided as defaults, and can be overwritten by passing the +corresponding keyword arguments to this function. + +Note: These parameters are initial estimates and should be calibrated using CalMIP protocol. +For calibration targets and observations, see the CalMIP Phase 1 protocol at: +https://github.com/callmip-org/Phase1 + +Data sources: +- Site characteristics from CalMIP Phase 1 protocol +- Initial parameter estimates based on deciduous broadleaf forest typical values +- Conductance parameters adapted from Wang et al. 2021 https://doi.org/10.5194/gmd-14-6741-2021 +""" +function FluxnetSimulations.get_parameters( + FT, + ::Val{:DK_Sor}; + # Soil hydraulic and thermal parameters + soil_ν = FT(0.50), + soil_K_sat = FT(5e-7), + soil_S_s = FT(1e-2), + soil_vg_n = FT(1.8), + soil_vg_α = FT(0.04), + θ_r = FT(0.05), + # Soil composition + ν_ss_quartz = FT(0.15), + ν_ss_om = FT(0.12), + ν_ss_gravel = FT(0.0), + # Soil surface properties + z_0m_soil = FT(0.01), + z_0b_soil = FT(0.01), + soil_ϵ = FT(0.98), + soil_α_PAR = FT(0.15), + soil_α_NIR = FT(0.15), + # Canopy structure parameters + Ω = FT(0.70), + χl = FT(0.1), + G_Function = ConstantGFunction(FT(0.5)), + # Leaf optical properties + α_PAR_leaf = FT(0.11), + λ_γ_PAR = FT(5e-7), + τ_PAR_leaf = FT(0.05), + α_NIR_leaf = FT(0.40), + τ_NIR_leaf = FT(0.25), + # Canopy radiative properties + ϵ_canopy = FT(0.97), + ac_canopy = FT(5e2), + # Photosynthesis and stomatal conductance parameters + g1 = FT(150), + Drel = FT(1.6), + g0 = FT(1e-4), + Vcmax25 = FT(6.5e-5), + # Water stress parameters + pc = FT(-2.0e6), + sc = FT(5e-6), + # Structural parameters + SAI = FT(1.0), + f_root_to_shoot = FT(3.0), + # Plant hydraulics parameters + K_sat_plant = 8e-8, + ψ63 = FT(-4 / 0.0098), + Weibull_param = FT(4), + a = FT(0.1 * 0.0098), + conductivity_model = PlantHydraulics.Weibull{FT}( + K_sat_plant, + ψ63, + Weibull_param, + ), + retention_model = PlantHydraulics.LinearRetentionCurve{FT}(a), + plant_ν = FT(1.44e-4), + plant_S_s = FT(1e-2 * 0.0098), + # Root and canopy height parameters + rooting_depth = FT(1.0), + n_stem = Int64(1), + n_leaf = Int64(1), + h_stem = FT(25), + h_leaf = FT(5), + h_canopy = h_stem + h_leaf, +) + return (; + soil_ν, + soil_K_sat, + soil_S_s, + soil_vg_n, + soil_vg_α, + θ_r, + ν_ss_quartz, + ν_ss_om, + ν_ss_gravel, + z_0m_soil, + z_0b_soil, + soil_ϵ, + soil_α_PAR, + soil_α_NIR, + Ω, + χl, + G_Function, + α_PAR_leaf, + λ_γ_PAR, + τ_PAR_leaf, + α_NIR_leaf, + τ_NIR_leaf, + ϵ_canopy, + ac_canopy, + g1, + Drel, + g0, + Vcmax25, + SAI, + f_root_to_shoot, + K_sat_plant, + ψ63, + Weibull_param, + a, + conductivity_model, + retention_model, + plant_ν, + plant_S_s, + rooting_depth, + n_stem, + n_leaf, + h_leaf, + h_stem, + h_canopy, + ) +end diff --git a/ext/fluxnet_simulations/data_processing.jl b/ext/fluxnet_simulations/data_processing.jl index 0684930f446..0d59a9a2046 100644 --- a/ext/fluxnet_simulations/data_processing.jl +++ b/ext/fluxnet_simulations/data_processing.jl @@ -163,9 +163,21 @@ end Reads Fluxnet CSV data for the provided site, and returns a tuple `(data, columns)` where `data` is a `n_rows x n_cols` Matrix of the data, and `columns` is a `1 x n_cols` Matrix of column names. + +This function now supports both standard Fluxnet sites and CalMIP sites. +For CalMIP sites (e.g., "DK-Sor"), it uses the callmip_data_path function. """ function read_fluxnet_data(site_ID) - fluxnet_csv_path = ClimaLand.Artifacts.experiment_fluxnet_data_path(site_ID) + # Check if this is a CalMIP site + callmip_sites = ("DK-Sor",) # Add more CalMIP sites here as needed + + if site_ID ∈ callmip_sites + fluxnet_csv_path = ClimaLand.Artifacts.callmip_data_path(site_ID) + else + fluxnet_csv_path = + ClimaLand.Artifacts.experiment_fluxnet_data_path(site_ID) + end + (data, columns) = readdlm(fluxnet_csv_path, ','; header = true) return (data, columns) diff --git a/ext/fluxnet_simulations/forcing.jl b/ext/fluxnet_simulations/forcing.jl index de4020e7738..9a63bf2a8d0 100644 --- a/ext/fluxnet_simulations/forcing.jl +++ b/ext/fluxnet_simulations/forcing.jl @@ -28,6 +28,11 @@ which is why we drop missing data. For the timestamp of the observation, we use the halfway point between the timestamp start and timestamp end: see https://fluxnet.org/data/fluxnet2015-dataset/fullset-data-product/ for details. +The `split_precip` keyword argument controls rain/snow partitioning: +- `true` or `:jennings`: Jennings et al. (2018) logistic function using temperature and humidity +- `:callmip`: CalMIP Phase 1 binary threshold (Tair < 273.15K → snow, >= 273.15K → rain) +- `false`: No splitting (all precipitation treated as rain) + This assumes that the first row of the CSV file is the list of of column names, and that these names are: "TA_F" (air temperature in C) @@ -153,7 +158,30 @@ function FluxnetSimulations.prescribed_forcing_fluxnet( # Convert to a flux using data_dt. Also, change the sign, because in ClimaLand # precipitation is a downward flux (negative), and convert from accumulated mm to m/s data_dt = Second(UTC_datetimes[2] - UTC_datetimes[1]).value # seconds - if split_precip + if split_precip == :callmip + # CalMIP Phase 1 protocol: binary threshold at 273.15K (0°C) + # Tair < 273.15K → all snow, Tair >= 273.15K → all rain + # TA_F is in °C, so threshold is 0°C + compute_rain_callmip(T, precip) = + T >= 0 ? -precip / 1000 / data_dt : zero(precip) + compute_snow_callmip(T, precip) = + T < 0 ? -precip / 1000 / data_dt : zero(precip) + atmos_P_liq = time_varying_input_from_data( + data, + ["TA_F", "P_F"], + column_name_map, + seconds_since_start_date; + preprocess_func = compute_rain_callmip, + ) + atmos_P_snow = time_varying_input_from_data( + data, + ["TA_F", "P_F"], + column_name_map, + seconds_since_start_date; + preprocess_func = compute_snow_callmip, + ) + elseif split_precip == true || split_precip == :jennings + # Jennings et al. (2018) logistic function using temperature and humidity compute_rain(T, VPD, precip; thermo_params = thermo_params) = -1 * precip / 1000 / data_dt * (1 - snow_precip_fraction(T, VPD; thermo_params)) @@ -366,6 +394,14 @@ Estimate the fraction of precipitation that is in snow form, given the air temperature at the surface in C and the vapor pressure deficit in hPa. +Uses a logistic function based on temperature and relative humidity, +which provides a smooth transition between rain and snow phases. +Used when `split_precip=true` or `split_precip=:jennings`. + +For CalMIP experiments, use `split_precip=:callmip` in +`prescribed_forcing_fluxnet` instead, which applies the CalMIP Phase 1 +binary threshold (Tair < 273.15K → snow, Tair >= 273.15K → rain). + See Jennings, K.S., Winchell, T.S., Livneh, B. et al. Spatial variation of the rain–snow temperature threshold across the Northern Hemisphere. Nat Commun 9, 1148 (2018). diff --git a/ext/fluxnet_simulations/initial_conditions.jl b/ext/fluxnet_simulations/initial_conditions.jl index 3900572755f..b74c9d53b11 100644 --- a/ext/fluxnet_simulations/initial_conditions.jl +++ b/ext/fluxnet_simulations/initial_conditions.jl @@ -47,7 +47,15 @@ function set_fluxnet_ic!( hour_offset_from_UTC, model::ClimaLand.AbstractLandModel, ) - fluxnet_csv_path = ClimaLand.Artifacts.experiment_fluxnet_data_path(site_ID) + # Check if this is a CalMIP site + callmip_sites = ("DK-Sor",) # Add more CalMIP sites here as needed + + if site_ID ∈ callmip_sites + fluxnet_csv_path = ClimaLand.Artifacts.callmip_data_path(site_ID) + else + fluxnet_csv_path = + ClimaLand.Artifacts.experiment_fluxnet_data_path(site_ID) + end # Read the data and get the column name map (data, columns) = readdlm(fluxnet_csv_path, ','; header = true) @@ -99,9 +107,9 @@ function set_fluxnet_ic!( ) FT = eltype(Y.soil.ρe_int) if isnothing(column_name_map["SWC_F_MDS_1"]) - θ_l_0 = model.parameters.ν / 2 + θ_l_0 = model.parameters.ν ./ 2 elseif unique(data[:, column_name_map["SWC_F_MDS_1"]]) == val - θ_l_0 = model.parameters.ν / 2 + θ_l_0 = model.parameters.ν ./ 2 else θ_l_0 = min.( diff --git a/src/Artifacts.jl b/src/Artifacts.jl index 0d616b367fc..89f3372284b 100644 --- a/src/Artifacts.jl +++ b/src/Artifacts.jl @@ -317,6 +317,49 @@ function experiment_fluxnet_data_path(site_ID; context = nothing) return data_path end +""" + callmip_data_path(site_ID; context = nothing) + +Return the path to the file that contains forcing data for a CalMIP site. +Currently supports DK-Sor (Denmark Sorø) for CalMIP Phase 1a test calibration. + +Site information: +DK-Sor (Denmark Sorø): +- Location: 55.486°N, 11.6446°E +- Vegetation: Deciduous Broadleaf Forest (Beech) +- CalMIP Phase 1a test site + +For more information see: https://github.com/callmip-org/Phase1 + +TODO: CalMIP data needs to be added to ClimaArtifacts. +For now, this expects the CSV files to be available locally. +Users should download CalMIP forcing data from: +- https://github.com/callmip-org/Phase1/tree/main/Data +- https://modelevaluation.org +""" +function callmip_data_path(site_ID; context = nothing) + @assert site_ID ∈ ("DK-Sor",) + + # TODO: Once CalMIP data is added to ClimaArtifacts, use: + # folder_path = @clima_artifact("callmip_sites", context) + # data_path = joinpath(folder_path, "$(site_ID).csv") + + # For now, expect users to provide the path via environment variable + # or use a default path + if haskey(ENV, "CALLMIP_DATA_PATH") + data_path = joinpath(ENV["CALLMIP_DATA_PATH"], "$(site_ID).csv") + else + # Default to sampo data directory + data_path = joinpath( + "/net/sampo/data1/renatob", + "callmip_forcing", + "$(site_ID).csv", + ) + end + + return data_path +end + """ esm_snowmip_data_path(; context = nothing)