Skip to content

feat: extend tyndp-archive with new datasets #758

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tgilon merged 4 commits into from
Jul 9, 2026
Merged

feat: extend tyndp-archive with new datasets #758

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
439b802
feat: extend tyndp-archive with new datasets
tgilon Jun 19, 2026
4184927
doc: add release note
tgilon Jun 19, 2026
aae1196
Merge branch 'merge-upstream-260617' into feat/extend-tyndp-archive
tgilon Jun 23, 2026
45a339d
Merge branch 'master' into feat/extend-tyndp-archive
tgilon Jul 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions config/data.tyndp.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,3 +195,12 @@ data:
open_tyndp_prelim:
source: tyndp-archive
version: latest
desnz_electricity_consumption:
source: tyndp-archive
version: latest
jrc_energy_atlas:
source: tyndp-archive
version: latest
ons_lad:
source: tyndp-archive
version: latest
6 changes: 5 additions & 1 deletion data/versions.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ cutout,v1.0,archive,latest supported,2026-01-13,Pre-build cutouts for PyPSA-Eur,
cutout,unknown,build,latest supported,2025-12-02,Build latest cutouts using ERA5/SARAH3 data.,
desnz_electricity_consumption,unknown,primary,latest supported,2026-02-02,,https://assets.publishing.service.gov.uk/media/6762951abe7b2c675de30705/Subnational_electricity_consumption_statistics_2005-2024.xlsx
desnz_electricity_consumption,2026-02-16,archive,latest supported,2026-02-16,,https://data.pypsa.org/workflows/eur/desnz_electricity_consumption/2026-02-16/Subnational_electricity_consumption_statistics_2005-2024.xlsx
desnz_electricity_consumption,2026-02-16,tyndp-archive,latest supported,2026-02-16,,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/desnz_electricity_consumption/2026-02-16/Subnational_electricity_consumption_statistics_2005-2024.xlsx
dh_areas,341.5,primary,latest supported,2025-12-02,,https://fordatis.fraunhofer.de/bitstream/fordatis/341.5/2/dh_areas.gpkg
dh_areas,341.5,archive,latest supported,2026-01-13,,https://data.pypsa.org/workflows/eur/dh_areas/341.5/dh_areas.gpkg
dh_areas,341.5,tyndp-archive,latest supported,2026-05-08,,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/dh_areas/341.5/dh_areas.gpkg
Expand Down Expand Up @@ -113,6 +114,7 @@ jrc_ardeco,2021,archive,latest supported,2026-01-13,,https://data.pypsa.org/work
jrc_ardeco,2021,tyndp-archive,latest supported,2026-05-08,,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/jrc_ardeco/2021
jrc_energy_atlas,unknown,primary,latest supported,2026-02-02,,https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/EIGL-Data/RASTER/electricity_tot_demand_2019.tif
jrc_energy_atlas,2026-02-16,archive,latest supported,2026-02-16,,https://data.pypsa.org/workflows/eur/jrc_energy_atlas/2026-02-16/electricity_tot_demand_2019.tif
jrc_energy_atlas,2026-02-16,tyndp-archive,latest supported,2026-02-16,,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/jrc_energy_atlas/2026-02-16/electricity_tot_demand_2019.tif
jrc_idees,2023-v1,primary,latest supported,2026-02-04,,https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/JRC-IDEES/JRC-IDEES-2023_v1/JRC-IDEES-2023.zip
jrc_idees,2023-v1,archive,latest supported,2026-02-09,,https://data.pypsa.org/workflows/eur/jrc_idees/2023-v1/JRC-IDEES-2023.zip
jrc_idees,2023-v1,tyndp-archive,latest supported,2026-05-08,,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/jrc_idees/2023-v1/JRC-IDEES-2023.zip
Expand Down Expand Up @@ -141,6 +143,7 @@ neso_electricity_demand,2026-02-02,archive,latest supported,2026-02-03,,https://
neso_electricity_demand,2026-02-02,tyndp-archive,latest supported,2026-05-08,,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/neso_electricity_demand/2026-02-02/electricity_demand_neso_raw.csv
nitrogen_statistics,2023,primary,latest supported,2025-12-02,,https://d9-wret.s3.us-west-2.amazonaws.com/assets/palladium/production/s3fs-public/media/files/myb1-2023-nitro-ERT.xlsx
nitrogen_statistics,2023,archive,latest supported,2026-02-26,,https://data.pypsa.org/workflows/eur/nitrogen_statistics/2023/myb1-2023-nitro-ERT.xlsx
nitrogen_statistics,2023,tyndp-archive,latest supported,2026-02-26,,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/nitrogen_statistics/2023/nitro-ert.xlsx
nitrogen_statistics,2022,primary,supported,2025-12-02,,https://d9-wret.s3.us-west-2.amazonaws.com/assets/palladium/production/s3fs-public/media/files/myb1-2022-nitro-ert.xlsx
nitrogen_statistics,2022,archive,supported,2026-01-13,,https://data.pypsa.org/workflows/eur/nitrogen_statistics/2022/nitro-ert.xlsx
nitrogen_statistics,2022,tyndp-archive,supported,2026-05-08,,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/nitrogen_statistics/2022/nitro-ert.xlsx
Expand All @@ -149,6 +152,7 @@ nuts3_population,13-03-2025,archive,latest supported,2026-01-13,earlier part of
nuts3_population,13-03-2025,tyndp-archive,latest supported,2026-05-08,earlier part of zenodo bundle,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/nuts3_population/13-03-2025/nama_10r_3popgdp.tsv.gz
ons_lad,may-2024,primary,latest supported,2026-02-02,API request used,https://services1.arcgis.com/ESMARspQHYMw9BZ9/arcgis/rest/services/Local_Authority_Districts_May_2024_Boundaries__UK_BSC/FeatureServer/0/query
ons_lad,may-2024,archive,latest supported,2026-02-17,API request used,https://data.pypsa.org/workflows/eur/ons_lad/may-2024/ons_lad.geojson
ons_lad,may-2024,tyndp-archive,latest supported,2026-02-17,API request used,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/ons_lad/may-2024/Local_Authority_Districts_May_2024_Boundaries__UK_BSC.geojson
open_tyndp_prelim,0.7.1,primary,not-supported,2026-06-02,Open-TYNDP preliminary outcomes,https://zenodo.org/records/20303009
open_tyndp_prelim,0.7.1,archive,latest supported,2026-06-02,Open-TYNDP preliminary outcomes (results-0.7.1.zip),https://zenodo.org/records/20303009/files/results-0.7.1.zip
open_tyndp_prelim,0.7.1,tyndp-archive,latest supported,2026-06-02,Open-TYNDP preliminary outcomes (results-0.7.1.zip),https://storage.googleapis.com/open-tyndp-data-store/outcomes/0.7.1/results-0.7.1.zip
Expand Down Expand Up @@ -210,7 +214,7 @@ swiss_energy_balances,2025-10-01,primary,latest supported,2026-02-12,,https://pu
swiss_energy_balances,2025-10-01,archive,latest supported,2026-02-16,,https://web.archive.org/web/20260216/https%3A//pubdb.bfe.admin.ch/de/publication/download/12361
synthetic_electricity_demand,v2,primary,latest supported,2025-12-02,the primary is already from Zenodo published by original authors,https://zenodo.org/records/10820928/files/demand_hourly.csv
synthetic_electricity_demand,v2,archive,latest supported,2026-01-20,the primary is already from Zenodo published by original authors,https://data.pypsa.org/workflows/eur/synthetic_electricity_demand/v2/demand_hourly.csv
synthetic_electricity_demand,v2,tyndp-archive,latest supported,2026-05-08,the primary is already from Zenodo published by original authors,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/synthetic_electricity_demand/v2
synthetic_electricity_demand,v2,tyndp-archive,latest supported,2026-05-08,the primary is already from Zenodo published by original authors,https://storage.googleapis.com/open-tyndp-data-store/archive/eur/synthetic_electricity_demand/v2/demand_hourly.csv
synthetic_electricity_demand,0.1.0,archive,deprecated might-work,2026-01-13,,https://data.pypsa.org/workflows/eur/synthetic_electricity_demand/0.1.0/demand_hourly.csv
tyndp,2024,primary,latest supported,2026-02-05,,https://2024-data.entsos-tyndp-scenarios.eu/files/scenarios-inputs
tyndp,2024,archive,latest supported,2026-02-05,,https://storage.googleapis.com/open-tyndp-data-store/2024
Expand Down
3 changes: 3 additions & 0 deletions doc/release_notes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@

* Fix CBA workflow to (a) release solver license after each successful rolling horizon optimization or after computing infeasibilities and (b) raise an error if rolling horizon fails when using HiGHS ([#756](https://github.com/open-energy-transition/open-tyndp/pull/756)).

* Extend `tyndp-archive` support to integrate three new datasets (`desnz_electricity_consumption`, `jrc_energy_atlas`, `ons_lad`) and two new versions (`nitrogen_statistics`, `synthetic_electricity_demand`) ([#758](https://github.com/open-energy-transition/open-tyndp/pull/758)).


**Documentation**

* Update benchmarking documentation tables and figures for v0.7.1 ([#711](https://github.com/open-energy-transition/open-tyndp/pull/711)).
Expand Down
6 changes: 3 additions & 3 deletions rules/retrieve.smk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -702,7 +702,7 @@ if (

if (ENERGY_ATLAS_DATASET := dataset_version("jrc_energy_atlas"))["source"] in [
"primary",
"archive",
*ARCHIVE_SOURCES,
]:

rule retrieve_electricity_demand_energy_atlas:
Expand All @@ -725,7 +725,7 @@ if (
DESNZ_ELECTRICITY_CONSUMPTION_DATASET := dataset_version(
"desnz_electricity_consumption"
)
)["source"] in ["primary", "archive"]:
)["source"] in ["primary", *ARCHIVE_SOURCES]:

rule retrieve_desnz_electricity_consumption:
output:
Expand All @@ -743,7 +743,7 @@ if (



if (ONS_LAD_DATASET := dataset_version("ons_lad"))["source"] in ["archive"]:
if (ONS_LAD_DATASET := dataset_version("ons_lad"))["source"] in [*ARCHIVE_SOURCES]:

rule retrieve_ons_lad:
input:
Expand Down