[Bug]: ImportError: cannot import name 'coupled_global'

[forsyth2]

What happened?

I ran into this issue while testing #29:

ImportError: cannot import name 'coupled_global' from 'zppy_interfaces.global_time_series.coupled_global'

Interestingly, I hadn't run into it at all on E3SM-Project/zppy#722 & #26 despite extensive testing of global_time_series in those PRs.

Full details

# Try 1: Unified works #########################################
cd ~/ez/zppy
git status
# No uncommitted changes
git fetch upstream main
git checkout -b test-gts-import-error upstream/main
conda clean --all --y
conda env create -f conda/dev.yml -n zppy-test-gts-import-error
conda activate zppy-test-gts-import-error
pip install .

# Edit tests/integration/utils.py
# UNIQUE_ID = "test-gts-import-error-try1"
#         "global_time_series_environment_commands": "source /lcrc/soft/climate/e3sm-unified/load_latest_e3sm_unified_chrysalis.sh",

# Edit v3 cfg to be as simple as possible
# By setting `active = False` as much as possible

python tests/integration/utils.py
pip install . && zppy -c tests/integration/generated/test_weekly_comprehensive_v3_chrysalis.cfg
cd /lcrc/group/e3sm/ac.forsyth2/zppy_weekly_comprehensive_v3_output/test-gts-import-error-try1/v3.LR.historical_0051/post/scripts && grep -v "OK" *status
# No errors

# Try 2: custom dev env fails #########################################
cd ~/ez/zppy-interfaces
git status
# No uncommitted changes
git fetch upstream main
git checkout -b test-gts-import-error upstream/main
conda clean --all --y
conda env create -f conda/dev.yml -n zi-test-gts-import-error
conda activate zi-test-gts-import-error
pip install .

cd ~/ez/zppy
conda activate zppy-test-gts-import-error
# Edit tests/integration/utils.py
# UNIQUE_ID = "test-gts-import-error-try2"
#         "global_time_series_environment_commands": "source /gpfs/fs1/home/ac.forsyth2/miniforge3/etc/profile.d/conda.sh; conda activate zi-test-gts-import-error",
python tests/integration/utils.py
zppy -c tests/integration/generated/test_weekly_comprehensive_v3_chrysalis.cfg
cd /lcrc/group/e3sm/ac.forsyth2/zppy_weekly_comprehensive_v3_output/test-gts-import-error-try2/v3.LR.historical_0051/post/scripts && grep -v "OK" *status
cat global_time_series_1985-1995.o786875

That gives:

Traceback (most recent call last):
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zi-test-gts-import-error/bin/zi-global-time-series", line 5, in <module>
    from zppy_interfaces.global_time_series.__main__ import main
  File "/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zi-test-gts-import-error/lib/python3.12/site-packages/zppy_interfaces/global_time_series/__main__.py", line 6, in <module>
    from zppy_interfaces.global_time_series.coupled_global import coupled_global
ImportError: cannot import name 'coupled_global' from 'zppy_interfaces.global_time_series.coupled_global' (/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zi-test-gts-import-error/lib/python3.12/site-packages/zppy_interfaces/global_time_series/coupled_global/__init__.py)

===== COPY FILES TO WEB SERVER =====

rsync: link_stat "/lcrc/group/e3sm/ac.forsyth2/zppy_weekly_comprehensive_v3_output/test-gts-import-error-try2/v3.LR.historical_0051/post/scripts/global_time_series_1985-1995_results" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1189) [sender=3.1.3]

What machine were you running on?

Chrysalis

Environment

See above

What command did you run?

See above

What stack trace are you encountering?

See above

[forsyth2]

I had also tried changing the name of the coupled_global function to run_coupled_global, but that had no effect.

[xylar]

@forsyth2, I think the problem might be that you have 2 different names for your conda environments that you're mixing up: zppy-test-gts-import-error and zi-test-gts-import-error. I would pick one and stick with it, and see if that fixes the problem.

[forsyth2]

@xylar Those are different environments. One is the zppy dev environment, the other is the zppy-interfaces dev environment.

[xylar]

@forsyth2, in your commands above, it looks like your mixing and matching them. And you really probably want one environment for co-developing zppy and zppy-interfaces together that includes all the dev dependencies for both. (However, this requires using conda create and the text-file specs for development requirements, since you can have --file <zppy_dev_specs> --file <zppy_interfces_dev_specs>, which is not possible with conda env create since that takes only one -f flag.

[tomvothecoder]

# Try 2: custom dev env fails #########################################
cd ~/ez/zppy-interfaces
git status
# No uncommitted changes
git fetch upstream main
git checkout -b test-gts-import-error upstream/main
conda clean --all --y
conda env create -f conda/dev.yml -n zi-test-gts-import-error
conda activate zi-test-gts-import-error
pip install .

In the zppy-interfaces dev environment above, try to isolate the environment and import statements below to see if it fails directly in zppy-interfaces.

Activate Python:

python 

Then:

import zppy_interfaces
from zppy_interfaces.global_time_series.coupled_global import coupled_global

[forsyth2]

Thanks @tomvothecoder

cd ~/ez/zppy-interfaces
git status
# No uncommitted changes
git fetch upstream main
git checkout -b issue-30-import-error upstream/main
lcrc_conda
conda clean --all --y
conda env create -f conda/dev.yml -n zi-issue-30-import-error
conda activate zi-issue-30-import-error
pre-commit run --all-files
git status
# No changes from pre-commit
pip install .

python

import zppy_interfaces
from zppy_interfaces.global_time_series.coupled_global import coupled_global
# Works fine!
from zppy_interfaces.global_time_series.__main__ import main
# Works fine!
exit()

python 

from zppy_interfaces.global_time_series.__main__ import main
# Works fine!

But setting up for a zppy Weekly run, I found:

cd ~/ez/zppy-interfaces
git fetch upstream main
git status
# Check for uncommitted changes
git checkout main
git reset --hard upstream/main
git log
# Check that latest commit matches https://github.com/E3SM-Project/zppy-interfaces/commits/main
# From 7/1: Update to latest conda-incubator/setup-miniconda 
# Matches, good
conda clean --all --y
conda env create -f conda/dev.yml -n zi-main-20250710
conda activate zi-main-20250710
pip install .
pytest tests/unit/global_time_series/test_global_time_series.py
# ImportError: cannot import name 'construct_generic_variables' from 'zppy_interfaces.global_time_series.coupled_global' (/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zi-main-20250710/lib/python3.12/site-packages/zppy_interfaces/global_time_series/coupled_global/__init__.py)

which isn't the same exact error, but is still an ImportError.

The full error is:

__________ ERROR collecting tests/unit/global_time_series/test_global_time_series.py ___________
ImportError while importing test module '/gpfs/fs1/home/ac.forsyth2/ez/zppy-interfaces/tests/unit/global_time_series/test_global_time_series.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
../../miniforge3/envs/zi-main-20250710/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/unit/global_time_series/test_global_time_series.py:5: in <module>
    from zppy_interfaces.global_time_series.coupled_global import (
E   ImportError: cannot import name 'construct_generic_variables' from 'zppy_interfaces.global_time_series.coupled_global' (/gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zi-main-20250710/lib/python3.12/site-packages/zppy_interfaces/global_time_series/coupled_global/__init__.py)

[forsyth2]

Intriguing, the conda environment is getting set up wrong. It's merging features.

> ls /gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zi-main-20250710/lib/python3.12/site-packages/zppy_interfaces/global_time_series/
classic                            coupled_global_viewer.py  __pycache__
coupled_global                     create_ocean_ts.py        utils.py
coupled_global_dataset_wrapper.py  index_template.html       viewer
coupled_global_plotting.py         __init__.py               zppy_land_fields.csv
coupled_global.py                  __main__.py
coupled_global_utils.py            ocean_month.py
> ls /gpfs/fs1/home/ac.forsyth2/miniforge3/envs/zi-main-20250710/lib/python3.12/site-packages/zppy_interfaces/global_time_series/coupled_global
driver.py            mix_viewer_component.py  mode_viewer.py      plotting.py
__init__.py          mix_viewer_original.py   plots_component.py  __pycache__
mix_pdf_original.py  mode_pdf.py              plots_original.py   utils.py

All that stuff in the second command is from #26, despite me building the conda environment off main...

[xylar]

Is the issue that a pip install . doesn't uninstall the old package first?

Would you consider installing in edit mode pip install -e .? To me, this is always more convenient for development.

Have you tried python -m pip install .? I doubt that would change anything but it's supposed to be the preferred approach.

Anyway, I'm mystified. Never seen this issue.

[forsyth2]

Is the issue that a pip install . doesn't uninstall the old package first?

That's what AI suggested, but I'm creating a brand new environment. Nevertheless, I tried it:

pip uninstall zppy-interfaces
# WARNING: Skipping zppy-interfaces as it is not installed.
pip install .
# This now fails!
# ERROR: Failed to build installable wheels for some pyproject.toml based projects (zppy_interfaces)

Full details

According to LivChat, "pip install . (and sometimes conda) does not always clean up removed files/directories from previous installs." Let's try again:

cd ~/ez/zppy-interfaces
git fetch upstream main
git status
# Check for uncommitted changes
git checkout main
git reset --hard upstream/main
git log
# Check that latest commit matches https://github.com/E3SM-Project/zppy-interfaces/commits/main
# From 7/1: Update to latest conda-incubator/setup-miniconda (#28)
# Matches, good
conda clean --all --y
conda env create -f conda/dev.yml -n zi-main-20250710-try2
conda activate zi-main-20250710-try2
pip uninstall zppy-interfaces
# WARNING: Skipping zppy-interfaces as it is not installed.
pip install .

gives:

Building wheels for collected packages: zppy_interfaces
  Building wheel for zppy_interfaces (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for zppy_interfaces (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [11958 lines of output]

[...]
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for zppy_interfaces
Failed to build zppy_interfaces
ERROR: Failed to build installable wheels for some pyproject.toml based projects (zppy_interfaces)

Try 3:

cd ~/ez/zppy-interfaces
git fetch upstream main
git status
# Check for uncommitted changes
git checkout main
git reset --hard upstream/main
git log
# Check that latest commit matches https://github.com/E3SM-Project/zppy-interfaces/commits/main
# From 7/1: Update to latest conda-incubator/setup-miniconda (#28)
# Matches, good
conda clean --all --y
conda env create -f conda/dev.yml -n zi-main-20250710-try3
conda activate zi-main-20250710-try3
pip install .

Same issue:

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for zppy_interfaces
Failed to build zppy_interfaces
ERROR: Failed to build installable wheels for some pyproject.toml based projects (zppy_interfaces)

Would you consider installing in edit mode pip install -e .?

lcrc_conda # reset conda, using alias
conda clean --all --y
conda env create -f conda/dev.yml -n zi-main-20250710-try4
conda activate zi-main-20250710-try4
pip install -e .
pytest tests/unit/global_time_series/test_global_time_series.py
# 10 passed in 26.83s

So, that actually works. I didn't know about that option. The problem though is if pip install -e . is just using the local files instead of the site-packages, then I can't switch branches while running it (i.e., to work on other things).

Have you tried python -m pip install .?

lcrc_conda # reset conda, using alias
conda clean --all --y
conda env create -f conda/dev.yml -n zi-main-20250710-try5
conda activate zi-main-20250710-try5
python -m pip install .
#   note: This error originates from a subprocess, and is likely not a problem with pip.
#   ERROR: Failed building wheel for zppy_interfaces
# Failed to build zppy_interfaces
# ERROR: Failed to build installable wheels for some pyproject.toml based projects (zppy_interfaces)

We're back to the the wheel building issue. So, now we can't even get to the ImportError step

[xylar]

@forsyth2, it shouldn't be trying to build a wheel. I think something must be misconfigured if that is happening.

Could you try this?

python -m pip install . --no-deps --no-build-isolation

That should prevent any attempts to install dependencies from PyPI instead of conda-forge. I don't see evidence that that's happening but just in case...

[xylar]

Also, the fact that you keep getting different errors is meta-odd. It feels like something bigger is wrong here.

[xylar]

Can you try this to get the full log?

pip install . -v | tee build_log.txt

Your error message suggests there are like 10,000 lines of error. Maybe there's a clue in there?