Skip to content

Add open-license energy balance sources - #543

Draft
Wegatriespython wants to merge 1 commit into
mainfrom
project/osce
Draft

Add open-license energy balance sources#543
Wegatriespython wants to merge 1 commit into
mainfrom
project/osce

Conversation

@Wegatriespython

Copy link
Copy Markdown
Contributor

Refiling of #539 This is a draft PR corresponding to the working branch for the osce-energy-resilience project. It adds support to fetch UNSD and EUROSTAT data to provide an alternative to IEA for energy balance data.

Add tools/unsd.py and tools/eurostat.py: national energy balances from the UN Statistics Division and from Eurostat over SDMX, each as a registered ExoDataSource and as a load_data() function for consumers that use pandas and have no Computer. Both emit the same dimensionality and units as IEA_EWEB under their own key tags, so a consumer can swap providers without its n and y dimensions changing shape.

Add util.sdmx.fetch_data() for the part of the query that is specific to neither service.

Neither aggregation nor interpolation runs unless asked for, and absent observations are never filled. The UNSD balance carries nine fuel groups against the IEA product axis of sixty-nine codes, so it serves aggregate indicators and cannot drive a technology calibration;

How to review

na

PR checklist

- [ ] Continuous integration checks all ✅

  • Add or expand tests; coverage checks both ✅
  • Add, expand, or update documentation.
    - [ ] Update doc/whatsnew.

Add tools/unsd.py and tools/eurostat.py: national energy balances from the UN
Statistics Division and from Eurostat over SDMX, each as a registered
ExoDataSource and as a load_data() function for consumers that use pandas and
have no Computer. Both emit the same dimensionality and units as IEA_EWEB under
their own key tags, so a consumer can swap providers without its n and y
dimensions changing shape.

Add util.sdmx.fetch_data() for the part of the query that is specific to
neither service.

Neither aggregation nor interpolation runs unless asked for, and absent
observations are never filled. The UNSD balance carries nine fuel groups
against the IEA product axis of sixty-nine codes, so it serves aggregate
indicators and cannot drive a technology calibration; the module docstring
says so.
@codecov

codecov Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.29078% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.5%. Comparing base (4f24a7e) to head (2da1998).

Files with missing lines Patch % Lines
message_ix_models/tools/eurostat.py 98.1% 1 Missing ⚠️
message_ix_models/tools/unsd.py 98.1% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main    #543     +/-   ##
=======================================
+ Coverage   74.3%   74.5%   +0.2%     
=======================================
  Files        323     327      +4     
  Lines      25850   26132    +282     
=======================================
+ Hits       19207   19488    +281     
- Misses      6643    6644      +1     
Files with missing lines Coverage Δ
message_ix_models/tests/tools/test_eurostat.py 100.0% <100.0%> (ø)
message_ix_models/tests/tools/test_unsd.py 100.0% <100.0%> (ø)
message_ix_models/tests/util/test_sdmx.py 100.0% <100.0%> (ø)
message_ix_models/util/sdmx.py 98.6% <100.0%> (+<0.1%) ⬆️
message_ix_models/tools/eurostat.py 98.1% <98.1%> (ø)
message_ix_models/tools/unsd.py 98.1% <98.1%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant