Tolerate missing PostgreSQL database in ixmp.testing

[khaeru]

#601 added code to the ixmp.testing.pytest_sessionstart() hook that connects to a PostgreSQL server for testing IXMP4Backend. The added code always runs, even if there is no PostgreSQL server available at the hard-coded URL.

This led to failures today in the scheduled message_ix, message-ix-models, and message_data workflows that use ixmp main:

INTERNALERROR> sqlalchemy.exc.OperationalError: (psycopg.OperationalError) connection failed: connection to server at "127.0.0.1", port 5432 failed: Connection refused
INTERNALERROR> 	Is the server running on that host and accepting TCP/IP connections?

To resolve:

• Wrap the connection step in a try:/except: block.
• Add a --ixmp-postgres pytest command-line option to control the connection parameters, in case downstream test suites wish to use different configuration.

Also:

• Don't parametrize tests for IXMP4Backend if no server is available.
• Show a message after collection.
• Per the SQLAlchemy docs, psycopg2 is the default engine for PostgreSQL. Use URLs starting with postgresql:// instead of postgresql+psycopg2:// where possible.

How to review

• Read the diff.
• Note that the CI checks all pass.
• Run a test suite that uses ixmp.testing as a plugin, e.g. the message-ix test suite.

PR checklist

• Continuous integration checks all ✅
• Add or expand tests; coverage checks both ✅
• Add, expand, or update documentation. Test changes only.
• Update release notes.

[codecov]

Codecov Report

❌ Patch coverage is 95.34884% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 98.3%. Comparing base (1bf6d34) to head (c92b9c7).
⚠️ Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
ixmp/testing/__init__.py	93.5%	2 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##            main    #611   +/-   ##
=====================================
  Coverage   98.2%   98.3%           
=====================================
  Files         51      51           
  Lines       6557    6577   +20     
=====================================
+ Hits        6444    6466   +22     
+ Misses       113     111    -2     

Files with missing lines	Coverage Δ
ixmp/_config.py	94.5% <ø> (ø)
ixmp/tests/test_tutorials.py	100.0% <100.0%> (ø)
ixmp/util/ixmp4.py	97.2% <100.0%> (+1.2%)	⬆️
ixmp/testing/__init__.py	94.1% <93.5%> (+1.1%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[khaeru]

The codecov/patch check failure leads me to see (here) that the pytest_sessionstart() hook is not run.

I'm not sure why this is, but in order to fix CI for other repos, will merge. We should investigate this later.

[glatterf42]

Thanks for these fixes and sorry that I didn't test message_ix against this branch (or note this requirement as a todo before merging).

[glatterf42]

I guess this line could have been removed now thanks to the new typeguard :)

[glatterf42]

I'm a bit confused about how we ensure the testing DB is running the latest state of the migrations, I thought setup() alone was not enough for that. But the passing tests indicate it works :)

[khaeru]

I looked for documentation for setup(), but it seems there isn't any. As far as I can guess, the setup() of a new backend sets it up using the schema corresponding to the ixmp4 version in use.

If the underlying database schema changes between ixmp4 versions, and the user accesses the same database with first one ixmp4 version and then a different version, then that package should do things like:

• apply migrations (from older to newer schemas, maybe in some configurable way), or
• raise appropriate exceptions (e.g. if it's not possible to go backwards from newer to older schemas)

The Java ixmp_source code does this. If the ixmp4 package itself doesn't do that yet, we can just advertise that as a limitation on using ixmp/message_ix with IXMP4Backend, for the time being. If it won't ever do that (considered out of scope), then it should be handled by IXMP4Backend itself, automatically and transparency, and shouldn't require any extra steps.

[glatterf42]

One more thing I just thought of here: we should make sure that no one accidentally runs the tests with WRITE permissions on a production DB to avoid deleting the whole DB. Though I'm not sure how we guard against that. Maybe once we have a URL for that, we can hardcode a check that this is not the DB we're about to drop?

[khaeru]

once we have a URL for that, we can hardcode a check that this is not the DB we're about to drop?

Yes, that sounds like a great safeguard. IOW before pytestconfig.options.ixmp_postgres is used, we can compare it to some fixed list and abort if it matches.

I think the production DBs should also be configured such that no users or very limited (admin) users have permission to use the SQL DROP DATABASE; and then tightly limit who can connect and authenticate as those users. That way, anyone (person or CI worker) trying to run the test suite will not have the right credentials/permissions, and this would fail even if they managed to connect.

Finally, we could also:

• Avoid using the name ixmp_test_.* for any production database. The code will only construct such names.
• Maybe change the constructed name to something like __ixmp_test, with the prefix underscore(s) indicating a non-production/special case.

That would be at least four kinds of protection.