Python package installations are affected by the use of `extension()` clause

[dipietrantonio]

Overview

Spack introduced a new clause when writing recipes for Python packages, the extends clause. You can read more about it here:

https://spack.readthedocs.io/en/latest/build_systems/pythonpackage.html#extends-vs-depends-on

Problem

This causes an issue because py-pip cannot install anymore in our user-specified location, but only in the environment location. Furthermore, any other py package loads the python-venv module and causes pip not to work properly.

Short term solution

I did some experimenting and I believe the quickest fix for our current setup is

1. Edit the pip and pip3 executables to use the "normal" python3 interpreter. For instance, I modified the pip executable on Joey like this:

cdipietrantonio@joey-02:/software/projects/pawsey0001/cdipietrantonio/pawsey-spack-config/scripts> cat `which pip`
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# old line
#!/software/setonix/2025.06/software/linux-sles15-zen3/gcc-14.2.0/python-venv-1.0-m4cdgxvvfzu74du7aflbz4wcshbebqet/bin/python3
import re
import sys
from pip._internal.cli.main import main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

2. comment out all the load('python-venv') in all Python modules' module files.

cdipietrantonio@joey-02:/software/projects/pawsey0001/cdipietrantonio/pawsey-spack-config/scripts> cat /software/setonix/2025.06/modules/zen3/gcc/14.2.0/python-packages/py-numpy/2.1.2.lua

[...]

--load(".gcc-runtime/14.2.0-lcqimel")
--load(".glibc/2.38-kq4yvye")
load("openblas/0.3.28")
load("python/3.11.6")
--load(".python-venv/1.0-2qual35")