Enable cloning from jdbc to ixmp4

[glatterf42]

In order to migrate from Oracle to PostgreSQL, we'll need a way to migrate (at least the most important) Scenarios. This PR adds a function that achieves just that, though there's still room for some improvement plus one open question.

@volker-krey In theory, you could use this branch to try cloning something from "ixmp_dev" to a local ixmp4/PostgreSQL DB (see iiasa/message_ix#981 on how to set one up).

More info:

• The branch depends on changes in the branch for Fixes for ixmp support September/October 2025 ixmp4#208, and can only be run with that version of ixmp4—not main, and not v0.14.0 from PyPI.
• Supersedes/replaces Implement clone() in Python #424.

Important question

• (Deferred to a subsequent PR.) Users might want to drop the timeseries/IAMC data that's considered "solution", but at least in the Python API, there's no way to read only timeseries data with meta=True. Should I try to hack something together using ixmp_source directly or are we okay with saying one can only clone full Scenarios from JDBC to ixmp4 (in that regard; dropping the optimization solution is easy)?
  • @khaeru: “It will be straightforward to provider users with utilities that can selectively remove time series data in scenarios that have been cloned to ixmp4 databases. That is, it can be a 2-step operation. Since the cross-backend clone will only happen in the context of careful migration at first, I think this will be fine.”

Possible improvements

• (Deferred.) Use bulk_insert() functions on the ixmp4 side: this is much more involved and requires implementing such functions in ixmp4 first, so if we don't mind performance so much, we could start an experiment with the current implementation.
  • @khaeru: “Given we are still waiting on merge of Fixes for ixmp support September/October 2025 ixmp4#208, let's not delay this PR for work that hasn't started yet upstream.”
• Add a docstring. Not sure this is needed, though, as the function should be self-explanatory (if long) and is marked as private.

How to review

• Read the diff and note that the CI checks (including the new test!) all pass.

PR checklist

• Continuous integration checks all ✅
• Add or expand tests; coverage checks both ✅
• Add, expand, or update documentation.
• Update release notes.

[khaeru]

I'll rebase this after the merge of #601.

[codecov]

Codecov Report

❌ Patch coverage is 99.33333% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 98.3%. Comparing base (d061bb0) to head (ab623ec).
⚠️ Report is 18 commits behind head on main.

Files with missing lines	Patch %	Lines
ixmp/core/scenario.py	98.0%	1 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff           @@
##            main    #610    +/-   ##
======================================
  Coverage   98.3%   98.3%            
======================================
  Files         51      51            
  Lines       6509    6624   +115     
======================================
+ Hits        6399    6518   +119     
+ Misses       110     106     -4     

Files with missing lines	Coverage Δ
ixmp/backend/base.py	99.5% <100.0%> (+<0.1%)	⬆️
ixmp/backend/common.py	100.0% <100.0%> (ø)
ixmp/backend/ixmp4.py	95.9% <100.0%> (+0.9%)	⬆️
ixmp/backend/jdbc.py	97.3% <100.0%> (ø)
ixmp/core/platform.py	98.9% <100.0%> (+<0.1%)	⬆️
ixmp/core/timeseries.py	97.6% <100.0%> (+0.1%)	⬆️
ixmp/testing/__init__.py	94.4% <100.0%> (+<0.1%)	⬆️
ixmp/tests/core/test_scenario.py	100.0% <100.0%> (ø)
ixmp/core/scenario.py	99.0% <98.0%> (-0.3%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[khaeru]

Thanks for the solid start here @glatterf42, especially for the test setup to confirm that the operation works.

I can join in/take over to make some adjustments. At a glance, I have these in mind:

• To align with existing separation-of-concerns, TimeSeries.clone() should handle clone of time-series data associated with the TimeSeries class, and Scenario.clone() should handle clone of model data. Each can make necessary adjustments on the target Platform for consistency.
• I think the proper location for clone code varies according to the situation:
  • Cloning within 1 Backend or across 2 Backends of the same type: should be in the respective backend module and make maximal use of low-level code for performance.
  • Cloning across 2 Backends of different type: should be high-level code using only the Platform, TimeSeries, and Scenario APIs, i.e. each Backend should not (need to) be aware that there is a distinct Backend class on the other end of the clone.

I see see the new high-level method Scenario.iter_item_data() is called from within a lower-level/underlying class, JDBCBackend. The latter adjustment would avoid having this somewhat backward relationship.

With the tests as a guardrail, I will try to make these changes as soon as I have a chance.

[khaeru]

Thanks @glatterf42 for developing this key feature and especially the tests for it, which made it easy to do some reorganization. This paves the way for similarly testing message_ix.Scenario.clone(): first to test databases, and eventually between centrally-operated IIASA Oracle and PostgreSQL databases.

At this point, it seems like ixmp per se is in good shape, and we are more waiting on:

1. Review and merge of iiasa/ixmp4#208 by maintainers.
2. Discussion on setting up and administering those central databases.
3. PRs for message_ix and message-ix-models.

…none of which should be in scope for the next week. So I will merge this PR, and then we can revise if necessary as those other pieces catch up. In particular, if the upstream PR is still pending by the time the 3.12 milestone comes (now scheduled 23 January), I can put in appropriate safeguard or warnings for the next release

[glatterf42]

Thanks for finishing this and I agree with you assessment. Only one note to 1.: as I understand it, iiasa/ixmp4#219 will pick up the changes from iiasa/ixmp4#208, too, so only the former might be merged. Though I'm not entirely sure on this, so we might have to watch out for both PRs.