Add .report.legacy.compat

[khaeru]

This PR adds 2 new modules:

• message_ix_models.report.legacy.compat.
  • This allows to add tasks to a Reporter ('new-style' or 'genno-based' reporting) that individually and collectively reproduce the behaviour of iamc_report_hackathon.main() ('existing' or 'legacy' reporting).
  • In particular the added run_table() function invokes a single legacy table as a single task in a Reporter.
• message_ix_models.report.legacy.config:
  • This handles the input data/configuration files needed to make the above work.

This addition is a prerequisite for the following steps that were recently discussed and planned:

1. Separate certain operations (merge_ts, merge_hist) of the 'legacy' reporting, so they can be re-used, replaced, improved, configured, etc.
2. Run a mix of legacy tables and new reporting code and integrate the results.

These steps should in turn make the 'legacy' reporting tables more user-friendly, allow some desired improvements, and ease co-use/integration/reimplementation in 'new' reporting.

Further details

• The implementation deliberately makes zero changes to code in iamc_report_hackathon, pp_utils, default_tables, et al. and the data files they read. Any compatibility code is contained in the new modules.
  • This should avoid complicating diffs as we update/merge/replace the existing files, e.g. with versions from message_data or message_single_country.
• This is not a recommended approach for new reporting code or features.
  • We will separately establish and document recommended approaches for writing new reporting code for new projects and model variants. The module is narrowly intended for the purposes mentioned above.
• There is an existing, older module with a similar name message_ix_models.report.compat.
  • The purpose of that module is different: it prepares a Reporter to exactly imitate the internal calculation steps done by a 'table' function. The module added in this PR instead patches the existing table functions into a Reporter as-is, and adds configuration so that they run correctly.
  • It was prepared as an experiment; see Report CO₂ emissions from transportation #134.
  • We've discussed that, for our ideal/future-proof reporting, it may be undesirable to exactly imitate the way calculations are performed in the existing tables. Thus the approach in message_ix_models.report.compat may not be applied widely.

Here is a visualization of the added tasks:

How to review

• Read the added docs, particularly for the function .report.legacy.compat.callback().
• Read the 1 added test in .tests.report.legacy.test_compat().
• (Maybe) Run the code against a full-scale scenario and check the results.
  • The scope for this PR does not include making this output 1:1 with iamc_report_hackathon for all combinations of arguments and inputs. It is narrowly intended to establish the basic functionality. However, such a check can identify TODOs for follow-up PRs.
• Note the CI checks all pass.

PR checklist

• Complete the
  • Basic implementation of merge_ts.
  • Basic implementation of merge_hist.
  • Patch complete IAMC-structured data into existing target keys that (a) write to (i) CSV, (ii) XLSX, (b) store on Scenario, or (c) some combination of these.
• Continuous integration checks all ✅
• Add or expand tests; coverage checks both ✅
• Add, expand, or update documentation.
• Update doc/whatsnew.

[codecov]

Codecov Report

❌ Patch coverage is 71.55963% with 62 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.5%. Comparing base (f498991) to head (e7bad98).

Files with missing lines	Patch %	Lines
message_ix_models/report/legacy/compat.py	24.6%	58 Missing ⚠️
...ssage_ix_models/tests/report/legacy/test_compat.py	88.0%	3 Missing ⚠️
message_ix_models/report/legacy/config.py	99.0%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##            main    #465     +/-   ##
=======================================
- Coverage   75.6%   74.5%   -1.1%     
=======================================
  Files        278     282      +4     
  Lines      22740   22957    +217     
=======================================
- Hits       17201   17125     -76     
- Misses      5539    5832    +293     

Files with missing lines	Coverage Δ
message_ix_models/report/operator.py	85.2% <100.0%> (ø)
...ssage_ix_models/tests/report/legacy/test_config.py	100.0% <100.0%> (ø)
message_ix_models/report/legacy/config.py	99.0% <99.0%> (ø)
...ssage_ix_models/tests/report/legacy/test_compat.py	88.0% <88.0%> (ø)
message_ix_models/report/legacy/compat.py	24.6% <24.6%> (ø)

... and 15 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.