Merge changes from upstream by tgilon · Pull Request #787 · open-energy-transition/open-tyndp

tgilon · 2026-07-08T09:35:37Z

Closes # (if applicable).

Changes proposed in this Pull Request

This PR brings the latest changes from PyPSA/PyPSA-Eur (up to PyPSA@ebd0cbe). Among other things, this PR introduces the following:

New guidelines for AI-based contributions (see original PR add AI contribution guidelines PyPSA/pypsa-eur#2215) and the documentation preview) ;
Updated package versions for PyPSA (v1.2.4), Linopy (v0.7.0) and Highs (v1.14.0 which does not yet include HiPO in python) ;
A configurable option to extend data/versions.csv (see original PR Make data/versions.csv easily extendable with a configurable option PyPSA/pypsa-eur#2016 and opened follow-up issue Disentangle data/versions.csv using data/tyndp_versions.csv #773) ;
A configurable default_target rule (see original PR Make default_target configurable PyPSA/pypsa-eur#2218) (it is suggested to keep all for now).

Merge conflicts:

.github/ISSUE_TEMPLATE/bug_report.yaml
.github/ISSUE_TEMPLATE/feature_request.md
doc/release_notes.md
rules/common.smk
rules/retrieve.smk
scripts/_helpers.py
test/test_data_versions_layer.py

Workflow

Open issues

Follow-up issues / PRs:

Notes

Checklist

Required:

Security scans show no high-severity bugs, critical vulnerabilities, or exposed secrets.
Changes are tested locally and behave as expected.
Code and workflow changes are documented.
A release note entry is added to doc/release_notes.md.

If applicable:

Changes in configuration options are reflected in scripts/lib/validation.
Changes in configuration options are added to config/test/*.yaml.
Multiple climate years test passes locally (pixi run -e open-tyndp tyndp-cyears-test).
For new data sources or versions, these instructions have been followed.
Open-TYNDP SPDX license header is added to all touched files.
Module docstrings are added to new Python scripts.
New rules are documented in the appropriate doc/*.md files.
Major features are documented in doc/index.md.

* fix: rename nice_names as well when clustering heat buses * add note

…#2202) * fix: keep unset p_set as NaN in cluster_heat_buses aggregation * fix: drop redundant numeric_only --------- Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

* doc: disable root toc entry * doc: add release note

* don't remove DC buses in sector network * bump DC nodes to low voltage, too * same bugfix in myopic code * restore master branch * add TODO note * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Bobby Xiong <36541459+bobbyxng@users.noreply.github.com> Co-authored-by: bobbyxng <bobbyxng@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

* fix: resolve plotting crashes due to missing tech colors Fixes PyPSA#2108. The plot summary and network mapping scripts crashed when technology color keys were missing from the plotting configuration. - Add missing 'heat dsm' color entry mapping - Implement _get_tech_colors() in plot_summary.py to safely lookup colors with warnings - Apply same safe color lookup and consistency logging to plot_power_network.py and plot_gas_network.py - Add test suite covering regression and safe fallback lookups * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * refactor: address review feedback, revert fallback colors and implement fail-fast validator * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * test: import check_tech_colors directly and remove duplicates --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

Co-authored-by: Lukas Trippe <lkstrp@pm.me>

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

…updates (PyPSA#2217) Bumps the github-actions group with 2 updates in the / directory: [actions/checkout](https://github.com/actions/checkout) and [actions/cache](https://github.com/actions/cache). Updates `actions/checkout` from 6 to 7 - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v6...v7) Updates `actions/cache` from 5 to 6 - [Release notes](https://github.com/actions/cache/releases) - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - [Commits](actions/cache@v5...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions - dependency-name: actions/cache dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* revert PyPSA#2201 PyPSA version limit * [pypsa-bot] run `sync-locks` - `pixi run sync-locks` --------- Co-authored-by: pypsa[bot] <181215446+pypsa[bot]@users.noreply.github.com> Co-authored-by: Lukas Trippe <lkstrp@pm.me>

…yPSA#2219) Bumps the github-actions group with 1 update: [prefix-dev/setup-pixi](https://github.com/prefix-dev/setup-pixi). Updates `prefix-dev/setup-pixi` from 0.9.6 to 0.10.0 - [Release notes](https://github.com/prefix-dev/setup-pixi/releases) - [Commits](prefix-dev/setup-pixi@v0.9.6...v0.10.0) --- updated-dependencies: - dependency-name: prefix-dev/setup-pixi dependency-version: 0.10.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…yPSA#2189) * Allow data version CSV file(s) to be specified in the config * Minor fixes * Allow YAML data versions * Cache loaded versions; refactor data version schema test * Revert config gen error message * fix: update error message to include all version files configured * Update docs * Path fixes --------- Co-authored-by: Johannes HAMPP <42553970+euronion@users.noreply.github.com> Co-authored-by: Lukas Trippe <lkstrp@pm.me>

* claude: fix deprecation warnings * pre-commit

Conflicts: .github/ISSUE_TEMPLATE/bug_report.yaml .github/ISSUE_TEMPLATE/feature_request.md doc/release_notes.md rules/common.smk rules/retrieve.smk scripts/_helpers.py test/test_data_versions_layer.py

daniel-rdt

I see only very minor merge conflicts this time in

Issue templates
rules/common.smk
scripts/lib/validation/config/data.py
Please correct me, if I am missing any. The above are looking good.
I think we can quickly bring this in.

Just to note that the grype check is still failing coming from upstream.

Question: Do we want to follow up and define our own default target rule?

tgilon · 2026-07-08T14:33:10Z

I see only very minor merge conflicts this time

Sorry, forgot to list them. I've just udpated the PR description.

Just to note that the grype check is still failing coming from upstream

Yes, it's failing for libssh2 but there is no more recent version exists atm

Do we want to follow up and define our own default target rule?

As I mentioned in the PR description, I don't think it's necessary. Do you have anything in mind?

daniel-rdt · 2026-07-08T14:37:58Z

Sorry, forgot to list them. I've just udpated the PR description.

Thanks, helpful indeed. Let me have another look then.

As I mentioned in the PR description, I don't think it's necessary. Do you have anything in mind?

We could consider cutting down on some of the upstream post-processing if we find not everything to be useful. But I don't think it is a high prio

daniel-rdt · 2026-07-08T14:54:37Z

Sorry, forgot to list them. I've just udpated the PR description.

Thanks, helpful indeed. Let me have another look then.

Ok, the other merge conflicts are also looking good. I actually noticed, that linopy is updated to 0.7.0 but not 0.8.0. Do you know why? It is older than 7d

tgilon · 2026-07-08T16:06:20Z

I actually noticed, that linopy is updated to 0.7.0 but not 0.8.0. Do you know why? It is older than 7d

The latest version on conda-forge is 0.7.0.

We could consider cutting down on some of the upstream post-processing if we find not everything to be useful. But I don't think it is a high prio

Unless you have a specific need for it, I suggest keeping things like this and revisiting once we have a more specific requirement.

Conflicts: envs/default_linux-64.pin.txt envs/default_osx-64.pin.txt envs/default_osx-arm64.pin.txt envs/default_win-64.pin.txt pixi.lock

lkstrp and others added 22 commits June 17, 2026 10:29

fix: permissions for lockfile update action (PyPSA#2205)

a4d522c

fix: diff view for SARIF report (PyPSA#2208)

1f8d4a5

fix: fix typo in rolling_horizon config of solve_network (PyPSA#2213)

f124355

fix: fix TYNDP typos (PyPSA#2214)

1363071

Fix: correctly rename nice_names when clustering heat buses (PyPSA#2203)

a235499

* fix: rename nice_names as well when clustering heat buses * add note

fix: keep unset p_set as NaN in cluster_heat_buses aggregation (PyPSA…

12610b3

…#2202) * fix: keep unset p_set as NaN in cluster_heat_buses aggregation * fix: drop redundant numeric_only --------- Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

doc: disable root toc entry (PyPSA#2216)

9ec7b65

* doc: disable root toc entry * doc: add release note

Make default_target configurable (PyPSA#2218)

cb11499

add AI contribution guidelines (PyPSA#2215)

7537e68

add highs-hipo solver settings set (PyPSA#2180)

76836f7

Co-authored-by: Lukas Trippe <lkstrp@pm.me>

make build_ambient_ait_temp script thread-safe (PyPSA#2197)

8e2068f

Co-authored-by: Fabian Neumann <fabian.neumann@outlook.de>

revert PyPSA#2201 PyPSA version limit (PyPSA#2204)

b9a248c

* revert PyPSA#2201 PyPSA version limit * [pypsa-bot] run `sync-locks` - `pixi run sync-locks` --------- Co-authored-by: pypsa[bot] <181215446+pypsa[bot]@users.noreply.github.com> Co-authored-by: Lukas Trippe <lkstrp@pm.me>

Fix future warnings (PyPSA#2220)

ebd0cbe

* claude: fix deprecation warnings * pre-commit

Merge branch 'PyPSA:master' into master

07176ae

Merge branch 'master-upstream-oet' into merge-upstream-260706

bc53922

Conflicts: .github/ISSUE_TEMPLATE/bug_report.yaml .github/ISSUE_TEMPLATE/feature_request.md doc/release_notes.md rules/common.smk rules/retrieve.smk scripts/_helpers.py test/test_data_versions_layer.py

refactor: solve merge conflicts

e82c258

chore: unpin linopy version

15d0001

tgilon added this to the Release v0.8 milestone Jul 8, 2026

tgilon self-assigned this Jul 8, 2026

tgilon added SB Scenario Building CBA Cost Benefit Analysis labels Jul 8, 2026

[github-actions.ci] Update locked envs

0a35079

tgilon marked this pull request as ready for review July 8, 2026 13:24

tgilon requested a review from daniel-rdt July 8, 2026 13:25

tgilon mentioned this pull request Jul 8, 2026

feat: disentangle data/versions.csv using data/tyndp_versions.csv #788

Merged

12 tasks

daniel-rdt approved these changes Jul 8, 2026

View reviewed changes

Merge branch 'master' into merge-upstream-260706

04d12f8

Conflicts: envs/default_linux-64.pin.txt envs/default_osx-64.pin.txt envs/default_osx-arm64.pin.txt envs/default_win-64.pin.txt pixi.lock

tgilon enabled auto-merge July 9, 2026 12:41

tgilon disabled auto-merge July 10, 2026 08:11

[github-actions.ci] Update locked envs

2c73f62

tgilon enabled auto-merge July 10, 2026 08:17

tgilon disabled auto-merge July 10, 2026 09:10

tgilon merged commit e52777f into master Jul 10, 2026
8 of 9 checks passed

tgilon deleted the merge-upstream-260706 branch July 10, 2026 09:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Merge changes from upstream - #787

Merge changes from upstream#787
tgilon merged 25 commits into
masterfrom
merge-upstream-260706

tgilon commented Jul 8, 2026 •

edited

Loading

Uh oh!

daniel-rdt left a comment •

edited

Loading

Uh oh!

tgilon commented Jul 8, 2026

Uh oh!

daniel-rdt commented Jul 8, 2026

Uh oh!

daniel-rdt commented Jul 8, 2026

Uh oh!

tgilon commented Jul 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

Uh oh!

Conversation

tgilon commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes proposed in this Pull Request

Workflow

Open issues

Notes

Checklist

Uh oh!

daniel-rdt left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tgilon commented Jul 8, 2026

Uh oh!

daniel-rdt commented Jul 8, 2026

Uh oh!

daniel-rdt commented Jul 8, 2026

Uh oh!

tgilon commented Jul 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

tgilon commented Jul 8, 2026 •

edited

Loading

daniel-rdt left a comment •

edited

Loading