Validation Health Status Workflow

[GbotemiB]

Closes # (if applicable).

The validation workflow in pypsa-earth-status evaluates solved PyPSA-Earth networks against real-world reference datasets (IRENA, and Our World in Data) to assess model accuracy and data quality.

Originally, the status check script was limited to validating a single network at a time.

This PR refactors the validation status workflow to support multi-scenario network validation, configurable reference sources with inclusion of Ember dataset covering demand, generation & installed capacity, and a health_status.csv that can help to collect validation results.

Changes proposed in this Pull Request

• Multi-scenario network validation: Support validating multiple resolved PyPSA networks (config.networks) with auto-detected or explicit country scopes.
• Dynamic reference data loading: Pull comparisons dynamically from IRENA (capacity), Our World in Data (demand), and Ember (generation) based on config.
• Dynamic versioning: Extract the pypsa-earth version directly from network metadata (n.meta) with a configurable config fallback.
• Global Tracker: The health_status.csv data will be used to collect validation results.
• Documentation: Documented network setup in README.md.

Verification & Testing

To test this implementation:

1. Configure solved network paths in config_copy.yaml:
   Add paths to your solved PyPSA network files (.nc) under the networks: block in config.yaml. You can use the simple string format (for auto-detecting countries) or the dictionary format (to specify explicit countries):

   network_validation:
     fallback_pypsa_earth_version: "v0.4.0"
     networks:
       NG_2021: "results/NG_2021/networks/elec_s_40flex_ec_lc1.0_1H.nc"
       ZA_2021:
         path: "results/ZA_2021/networks/elec_s_40flex_ec_lc1.0_1H.nc"
         countries: ["ZA"]

2. Configure multiple sources in config_copy.yaml:
   Configure multiple sources for demand to verify that the long-format scales cleanly:

   datasets:
     demand: ["ourworldindata", "ember"]
     installed_capacity: ["irena"]
     generation: ["ember"]

3. Run the Snakemake workflow:

   snakemake -j 1 results/health_status.csv

4. Inspect the output (results/health_status.csv):

Checklist

• I consent to the release of this PR's code under the AGPLv3 license and non-code contributions under CC0-1.0 and CC-BY-4.0.
• I tested my contribution locally and it seems to work fine.
• Code and workflow changes are sufficiently documented, including updates to docstrings for meaningful functions.
• Newly introduced dependencies are added to envs/environment.yaml.
• Changes in configuration options are added in all of config.yaml.
• A note for the release notes doc/release-notes.md is amended in the format of previous release notes, including reference to the requested PR.

[davide-f]

That is great @GbotemiB ! Give me a few days for the review. Looks amazing!

[davide-f]

Thanks Emmanuel, nice introductions! Ember and Health status! :D

[davide-f]

I see you have worked in parallel to Daniele on the generation mix. Please @danielelerede-oet and @GbotemiB , look at each other PRs and feel free to propose a common strategy.
Emmanuel used a different data source (Ember) and Daniele IRENA, so both are highly relevant features

[GbotemiB]

Thanks @davide-f, @danielelerede-oet, here's what I'm seeing comparing #61 and #63:

• We both independently fixed the same thing: accounting for storage-unit capacity in installed/optimal capacity stats, and computing network-side generation from generators_t.p + storage_units_t.p. These will conflict directly if merged separately.
• Different generation reference sources: I used Ember, you used IRENA. My build_reference_statistics.py already treats each dataset type (demand, installed_capacity, generation) as a configurable list of sources (datasets.generation: [...]), so IRENA could slot in as a second source alongside Ember using the same pattern, once I address Davide's ask to harmonize columns instead of branching per-source.
• Add electricity generation validation based on IRENA data #63 adds the downstream wiring (make_comparison.py, visualize_data.py) to actually compare/plot generation, which Validation Health Status Workflow #61 doesn't have yet. Add electricity generation validation based on IRENA data #63 also carries Store validation outputs per configuration #62's config-specific output directories, which my multi-network support will need anyway.

Proposal: land #62 first (output dirs), then converge on one generation-mix implementation, my multi-source datasets.generation config as the interface (supporting both ember and irena), your IRENA cleaning script + comparison/plotting wiring, and a single shared fix for the storage-capacity accounting.

Happy to hop on a call to divide up the remaining work if that's easier than doing it over PR comments.

[davide-f]

Works for me. We focus on #62 first. What about #63? Can we merge that and revisit here?
Please @GbotemiB and @danielelerede-oet share your view, given your possibilities

[GbotemiB]

Agree to merge #63, then revisit here.

[davide-f]

At this stage, I wouldn't expect to have different behaviour depending on the source, that would mean that the prior rules do have different data format. Can we harmonize the column names of the new sections to keep the common format?

In the else case, please add a warning highglighting empty demand

[davide-f]

In the future, here we could have
demand = [....], where the list is populated dynamically depending on the config option, then the code can be modularized even further.
I'm adding an issue on this, no action