Use of environment variables in `modextrapaths` and `modextravars`

[branfosj]

When using Tcl module files and Lmod we cannot have environment variables defined in one module file being used in another module file ($EBROOTXXX being used by YYY for example). Environment variables that are known outside of module files are fine ($HOME for example) and so are environment variables used in the same module file they are defined ($EBROOTXXX being used by XXX for example). The reason for this is described at https://lmod.readthedocs.io/en/latest/095_tcl2lua.html. An example build failure is in #23119 (comment), with error:

Lmod has detected the following error:
/tmp/eb-euvuahcw/tmpvhgy06te/all/Triton/2.3.1-foss-2023b-CUDA-12.4.0:
(Triton/2.3.1-foss-2023b-CUDA-12.4.0): can't read "::env(CUDA_HOME)": no such
variable 

Problems

funannotate-1.8.17-foss-2023a.eb (#24065)

modextravars = {
    'GENEMARK_PATH': '$EBROOTGENEMARKMINET',
}

PAPI-7.2.0-GCCcore-14.2.0-CUDA-12.8.0.eb and PAPI-7.2.0-GCCcore-14.3.0-CUDA-12.9.1.eb

modextravars = {
    'PAPI_CUDA_ROOT': '$EBROOTCUDA',
}

spacyr-1.3.0-foss-2023a.eb

modextravars = {
    'RETICULATE_PYTHON': '$EBROOTPYTHON/bin/python',
    'SPACY_PYTHON': '$EBROOTSPACY',
}

Triton-2.1.0-foss-2023a-CUDA-12.1.1.eb

modextravars = {
    'TRITON_PTXAS_PATH': '$CUDA_HOME/bin/ptxas',
}

Triton-2.3.1-foss-2023b-CUDA-12.4.0.eb

modextravars = {
    # Search for "_path_to_binary" in Triton sources
    'TRITON_CUOBJDUMP_PATH': '$CUDA_HOME/bin/cubjdump',
    'TRITON_LIBDEVICE_PATH': '$CUDA_HOME/nvvm/libdevice/libdevice.10.bc'
}

Triton-3.1.0-foss-2024a-CUDA-12.6.0.eb

modextravars = {
    'TRITON_PTXAS_PATH': '$CUDA_HOME/bin/ptxas',
    # ensure that libdevice.10.bc from $EBROOTCUDA/nvvm/libdevice is used:
    'TRITON_LIBDEVICE_PATH': '$CUDA_HOME/nvvm/libdevice/libdevice.10.bc'
}

Not problems

• JupyterLab uses $HOME.
• mujoco-py-2.1.2.14-foss-2021b.eb uses $EBROOTMUJOCO is fine as the environment variable is defined in the module file it is used
• Schrodinger uses $HOME

[Micket]

Well. this sure is unfortunate. It basically means that apart from literally just some examples using $HOME, the easybuilders/easybuild-framework#4855 was basically useless effort then? At least for any upstream easyconfigs. ( ping @lexming )

[lexming]

Not useless, this issue is specific to Tcl+Lmod and its on-the-fly conversion of Tcl to Lua. So Lua+Lmod and Tcl+EnvMod are fine, right?

For some cases like funannotate/Genemark an ugly workaround could be to move the definition of those envars to the parent module. But that is not applicable everywhere.

As explained in Lmod docs, the only real solution is to switch to Lua for these easyconfigs. So maybe the only thing we can do on EB side is to print a warning for these modules if we detect the fatal Tcl+Lmod combination. Users in that situation can easily switch to Lua for these few modules as Lmod can work with a mixture of Tcl and Lua.