Test failure in PR #326: "No source found that can handle 'ICONICS:SSP(2024).1'" (Commit f886d2d9a4f96b3d9ad844ac216b531c25a2c47b)

[Wegatriespython]

Title:
Test failures in PR #326: "No source found that can handle 'ICONICS:SSP(2024).1'" (Commit f886d2d)

Description:
After merging PR #326 (specifically commit f886d2d9a4f96b3d9ad844ac216b531c25a2c47b), tests in both test_gdp.py and test_projections.py started failing. The core issue in both files appears to be the inability to handle the source identifier 'ICONICS:SSP(2024).1', resulting in the following error:

ValueError: No source found that can handle 'ICONICS:SSP(2024).1'

Failure Details (test_projections.py):

Running pytest message_ix_models/tests/tools/costs/test_projections.py results in:

• 7 Failed Tests:
  • test_create_cost_projections[config0-exp_fix0-exp_inv0] (SSP2, R11)
  • test_create_cost_projections[config2-exp_fix2-exp_inv2] (SSP5, R12, cooling)
  • test_bare_res[R11]
  • test_bare_res[R12]
  • test_ccs_costs[energy]
  • test_ccs_costs[materials]
  • test_ccs_costs[cooling]
• 2 XFailed Tests (Expected Failures):
  • test_create_cost_projections with Config(node="R20")
  • test_bare_res[R20]
• 1 Passed Test:
  • test_create_cost_projections[config1-exp_fix1-exp_inv1] (SSP2, materials, convergence) - Note: This test seems unaffected as it uses the 'convergence' method, not 'gdp'.

Failure Details (test_gdp.py):

Running pytest message_ix_models/tests/tools/costs/test_gdp.py results in all 5 tests failing with the same ValueError.

Failure Trace (Common to both files):
The error originates when calling prepare_computer from within process_raw_ssp_data:

..\..\..\tools\costs\gdp.py:74: in process_raw_ssp_data
    key, *_ = prepare_computer(context, c, ssp, source_kw, strict=False)

.\..\..\tools\exo_data.py:231:

context = <Context object ...>, c = <genno.core.computer.Computer object ...>
source = 'ICONICS:SSP(2024).1', source_kw = {'measure': 'POP', 'model': 'IIASA-WiC POP 2023', 'name': '_pop 1'}

# ... (code for prepare_computer) ...

    if source_obj is None:
>       raise ValueError(f"No source found that can handle {source!r}")
E       ValueError: No source found that can handle 'ICONICS:SSP(2024).1'

..\..\..\tools\exo_data.py:231: ValueError

Context:
When resetting main to the commit before the problematic one (e.g., git reset --hard dadf8d891), the tests in both files pass again.

git bisect identified the first bad commit as:

f886d2d9a4f96b3d9ad844ac216b531c25a2c47b is the first bad commit
commit f886d2d9a4f96b3d9ad844ac216b531c25a2c47b
Author: Paul Natsuo Kishimoto <mail@paul.kishimoto.name>
Date:   Fri Apr 4 11:16:11 2025 +0200

    Simplify path handling in SSP{Original,Update}

    Use path_fallback() in both methods with common arguments.

 message_ix_models/project/ssp/data.py | 60 ++++++++++++++---------------------
 1 file changed, 23 insertions(+), 37 deletions(-)

This commit simplified path handling in message_ix_models/project/ssp/data.py, which seems linked to this regression.

Reproduction:

1. Check out commit f886d2d9a4f96b3d9ad844ac216b531c25a2c47b or any later commit on main.
2. Run the tests:

   pytest message_ix_models/tests/tools/costs/test_gdp.py
   pytest message_ix_models/tests/tools/costs/test_projections.py

3. Observe the failures detailed above.

Environment:

• OS: Windows
• Python: 3.13.2
• Test framework: pytest-8.3.5

[khaeru]

Hi @Wegatriespython, some information is missing from this report that would be useful.

• Does this occur when running pytest, or something else? If pytest, which test(s) fail(s)?
• On your system, or elsewhere? What is the environment?
• The "Failure Trace" is only fragmentary. In which file is this code? Do you have a full traceback?

More to the point, while it is good to identify that this occurs after a commit from #326, the next step is to look at the PR description/contents and understand what changed. In particular, the description contains:

How to review

Read the following updated documentation pages and ensure instructions are clear:

• …
• HOWTO Work with paths to files and data

This page contains a new section “HOWTO Connect static data to local data“ which I might guess is related to your issue.

Did you see this or try it?

[Wegatriespython]

Thank @khaeru, Ill check it once and get back if the issue is further upstream or can be locally resolved within test_gdp.py

[Wegatriespython]

I made some additional changes to error handling in util path_fallback and tools\exo_data to treat filenotfound error seperate from valueerrors, this gave me a more informative error message.

FAILED test_gdp.py::test_process_raw_ssp_data[R11] - FileNotFoundError: 'ssp\1706548837040-ssp_basic_drivers_release_3.0_full.csv.gz' not found in any of [WindowsPath('C:/Users/vigne/...
FAILED test_gdp.py::test_process_raw_ssp_data[R12] - FileNotFoundError: 'ssp\1706548837040-ssp_basic_drivers_release_3.0_full.csv.gz' not found in any of [WindowsPath('C:/Users/vigne/...
FAILED test_gdp.py::test_adjust_cost_ratios_with_gdp[energy] - FileNotFoundError: 'ssp\1706548837040-ssp_basic_drivers_release_3.0_full.csv.gz' not found in any of [WindowsPath('C:/Users/vigne/...
FAILED test_gdp.py::test_adjust_cost_ratios_with_gdp[materials] - FileNotFoundError: 'ssp\1706548837040-ssp_basic_drivers_release_3.0_full.csv.gz' not found in any of [WindowsPath('C:/Users/vigne/...
FAILED test_gdp.py::test_adjust_cost_ratios_with_gdp[cooling] - FileNotFoundError: 'ssp\1706548837040-ssp_basic_drivers_release_3.0_full.csv.gz' not found in any of [WindowsPath('C:/Users/vigne/...

The primary cause of this error detection is the use of path_fallback

New Version

class SSPUpdate(ExoDataSource):
#rest of the code 
        self.path = path_fallback("ssp", self.filename[release], where=WHERE)
#rest of the code

Old version

class SSPUpdate(ExoDataSource):
#rest of the code 
        # Iterate over possible locations for the data file
        for path in [d.joinpath(self.filename[release]) for d in dirs]:
            if not path.exists():
                log.info(f"Not found: {path}")
                continue
            if "test" in path.parts:
                log.warning(f"Reading random data from {path}")
            break

        self.path = path # potentially non-existent path
#rest of the code

Now I don't understand the files themselves well enough or how patch fixes for test work so I am not sure how to proceed. Would appreciate any guidance @khaeru

[khaeru]

Please (re-)read my last comment, especially the final paragraphs.