feat: added scatter summary plot per horizon for all indicators

[AndreasHD11]

Closes #685 #712

Changes proposed in this Pull Request

This PR adds a scatter plot per planning horizon, for each indicator for all projects. The plot compares TYNDP and Open-tyndp weighted average values for each indicator.

Workflow

1. To test, I downloaded the NT-cyears-20260425 folder as it contained all the ensemble indicators for each project (cy weighted average values per project).

2. I ran: snakemake --configfile config/config.tyndp.yaml --cores 1 "results/tyndp/NT-cyears/cba/ensemble_plots/summary_benchmark_2030.png" "results/tyndp/NT-cyears/cba/ensemble_plots/summary_benchmark_2040.png" --allowed-rules plot_summary_projects_benchmark --rerun-triggers mtime

3. In ensemble_plots folder:
   2030

2040

Notes

1. In config/config.tyndp.yaml, I ran run.name: "NT-cyears". For the projects, I ran in cba.projects: t1-t1300 (minus t339 as this was creating issues).
2. Colour scheme is kept for b2a_societal_cost indicator to better see the low/central/high scenarios.

Changes

1. New rule in cba.smk called rule plot_summary_projects_benchmark
2. New plotting function in plot_benchmark_indicators.py called plot_summary_projects_benchmark

Checklist

Required:

• [x ] Changes are tested locally and behave as expected.
• [x ] Code and workflow changes are documented.
• A release note entry is added to doc/release_notes.md.

If applicable:

• Changes in configuration options are reflected in scripts/lib/validation.
• OET SPDX license header added to all touched files.
• For new data sources or versions, these instructions have been followed.
• New rules are documented in the appropriate doc/*.md files.

[measrainsey]

Thank you @AndreasHD11 for implementing this feature! It's really helpful for getting an overview of how our CBA indicators look compared to TYNDP 2024 :)

I can confirm that it works on my end when I run the CBA workflow with run.name = "NT-cyears" and cba.projects: [-t339] (the t339 problem is being fixed in #750), the following files are generated for me:

• results/tyndp/NT-cyears/cba/ensemble_plots/summary_benchmark_2030.png
• results/tyndp/NT-cyears/cba/ensemble_plots/summary_benchmark_2040.png

Can I make two requests:

1. I don't think these summary benchmark plots are generated when running a scenario like NT (I tested). Of course in these cases we would have less data points to plot, but I think we would want that functionality, since we do a lot of checking results with just the NT scenario and not always with a collection scenario like NT-cyears. Would it be possible to modify your implementation to work also with single scenarios like NT, NT-cy2009, etc?
2. Is it possible to have different diagonal lines for the b2a_societal_cost indicator (one for low/med/high, in the same colors as the dots), instead of one shared diagonal line?

[AndreasHD11]

I ran it again with the new changes with the following config and commands:

• For run.name = "NT-cyears" and projects t1-t1400(removing t339 as not yet fixed in this branch) With command:snakemake --configfile config/config.tyndp.yaml --cores 1 "results/tyndp/NT-cyears/cba/ensemble_plots/summary_benchmark_2030.png" "results/tyndp/NT-cyears/cba/ensemble_plots/summary_benchmark_2040.png" --allowed-rules plot_summary_projects_benchmark --rerun-triggers mtime`

Using results from: NT-cyears-20260425
Example:
2030:

2040:

I also ran it with more recent results and only one climate year such as NT-cy2009:
With command:
snakemake --configfile config/config.tyndp.yaml --cores 1 "results/tyndp/NT-cy2009/cba/ensemble_plots/summary_benchmark_2030.png" "results/tyndp/NT-cy2009/cba/ensemble_plots/summary_benchmark_2040.png" --allowed-rules plot_summary_projects_benchmark --rerun-triggers mtime
Using results from: NT-cy2009-20260528-co2seq-1e6-gurobi

2030:

2040:

I also tested with just NT for run.name and it also worked.

[AndreasHD11]

For this: Is it possible to have different diagonal lines for the b2a_societal_cost indicator (one for low/med/high, in the same colors as the dots), instead of one shared diagonal line?

The diagonal lines are just comparison y=x lines at 45 degrees. So when a dot (indicator value of project t) is on the line it means the TYNDP and Open-TYNDP values are equal. If it's higher then our values are higher than TYNDP for example. Having a diagonal line for each scenario in b2a would just be 3 superimposed lines I'm guessing. Could you clarify if the question is different to my answer? Thanks!

[lisazeyen]

@AndreasHD11 awesome, thanks for preparing! The plots look really nice.

A few points to consider:

add overall indicator

What I am missing is an overall value for each indicator which shows us if we are moving closer to the official TYNDP indicators. I would suggest to
1.) calculate an indicator similar to SB benchmarking plot_benchmark.py
2.) add this value to each plot as a text, similar to here

societal emission costs

I think we can just compare the central level for the benchmarking, the other two levels should be related, since it is a multiplication of costs*emissions, and only the emissions are part of the optimisation

merge conflicts

Merge conflicts need to be resolved.

[AndreasHD11]

Removing the low and high scenarios (and colours) for b2a, and adding a box with the n, sMAPE and sMdAPE for each indicator:
Using the same command as above for NT-cyears

2030:

2040:

[AndreasHD11]

Am fixing failed test for cba.smk

[AndreasHD11]

The output of the rule initially did not include a run wildcard, but the rule asked for a run wildcard, which means that it worked locally for me as run.scenarios was enabled. But in the test it would fail.

Meas is on holidays, LZ will review instead

[lisazeyen]

Great, thanks a lot @AndreasHD11 ! Good to go!

[tgilon]

Thank you for the feature! One small suggestion, I think it would help to have equal axis on the plots. This would make them easier to read. It's too late for this PR, but it could be added later.