Dockerfile partially broken. Nonexistant python3

[DagurB]

Steps To Reproduce

$docker run -it sagemath/sagemath:10.7 /bin/bash
sage@7f620c268e9a:~$ sage
/usr/bin/env: ‘python3’: No such file or directory
sage@7f620c268e9a:~$ sage -pip install pycryptodome
/home/sage/sage/src/bin/sage: line 599: exec: python3: not found

Notice that sage doesn't work due to python3 apparently not existing. Running the image normally (i.e. without /bin/bash) works though.

Expected Behavior

For sage to launch, and for sage -pip to work

Actual Behavior

Sage throws an error, stating that python3 doesn't exist.

Additional Information

Some discussion in Cryptohack's #sagemath-dev-chat lead to speculations about 97d834b being the culprit, but I don't really know how to verify that.

Environment

• OS: Void Linux
• Sage Version: 10.7 (docker)

Checklist

• I have searched the existing issues for a bug report that matches the one I want to file, without success.
• I have read the documentation and troubleshoot guide

[user202729]

@tobiasdiez @antonio-rojas any idea?

[antonio-rojas]

Seems to me that /sage/local/var/lib/sage/venv-python3.12.5/bin/ should be in the docker's PATH, but somehow it isn't.

[user202729]

I guess adding that to PATH (e.g. by activating the venv) may work.

But there remains the underlying that why were sage-python wrapper introduced in the first place, if one could "just" activate the venv. It was probably introduced for some reason.

[JeanPaulShapo]

Actually, the PATH environment variable is initially set correctly and includes
/sage/local/var/lib/sage/venv-python3.12.5/bin/, but it is later overridden by ~/.bashrc when /bin/bash is launched.

(In this comment, I use the 10.8 tag as the most recent stable release at the time of writing.)

Suppose we run

docker run -it sagemath/sagemath:10.8 /bin/bash

What is happening internally?

1. From the Dockerfile, we can see what is executed when a Sage Docker container starts:
   https://github.com/sagemath/sage/blob/10.8/docker/Dockerfile#L237-L241
   From this, we can deduce that the actual command being run is /usr/local/bin/sage-entrypoint /bin/bash (ENTRYPOINT plus overridden CMD)
2. According to this COPY statement /usr/local/bin/sage-entrypoint corresponds to https://github.com/sagemath/sage/blob/10.8/docker/entrypoint.sh. With /bin/bash as an argument, it eventually executes https://github.com/sagemath/sage/blob/10.8/docker/entrypoint.sh#L8, which expands to exec sage -sh -c "/bin/bash" in this case
3. The sage executable turns out to be src/bin/sage script (I unfortunately do not have a short explanation for how this indirection occurs). While the script is quite large, two parts are particularly relevant:
   • At https://github.com/sagemath/sage/blob/10.8/src/bin/sage#L254-L266, the
     sage-env script is invoked. This script prepends directories to PATH in https://github.com/sagemath/sage/blob/10.8/src/bin/sage-env#L254-L262
   • At https://github.com/sagemath/sage/blob/10.8/src/bin/sage#L697-L828, arguments for executing Bash are prepared
     First, SHELL_OPTS is set to --norc (see https://github.com/sagemath/sage/blob/10.8/src/bin/sage#L743), and then exec "$SHELL" $SHELL_OPTS "$@" is executed
4. This ultimately expands to exec "/bin/bash" "--norc" "-c" "/bin/bash"
   This is where the subtle issue arises. There are three relevant stages where environment variables could be affected:
   • exec itself does not modify the environment; it merely replaces the current process
   • /bin/bash --norc -c ... also behaves as expected, since --norc prevents reading ~/.bashrc
   • the second /bin/bash invocation is the problem - it is started without --norc, so it reads ~/.bashrc. In that file, there is a line that overrides PATH (added during image build here)

At this point, it should be clear why python3 no longer works as expected.
But what about the sage command itself? Should it not go through the same environment setup?

The key detail is that sage-env is designed to be re-entrant. It contains checks that prevent re-setting environment variables if they are already marked as initialized. Since SAGE_ENV_SOURCED was set during the earlier /bin/bash invocation, the environment is not reconfigured.

This leaves us with several possible ways to address the issue. Below are three options, ordered from most to least favorable (in my opinion):

1. Add --norc to the docker run command - docker run -it sagemath/sagemath:10.8 /bin/bash --norc
   • good - the environment variables remain correct
   • bad - this changes the user-facing command
   • ugly - PS1 is reset, resulting in a bash prompt like bash-5.1$
2. Use /bin/bash as the container entrypoint - docker run --entrypoint /bin/bash -it sagemath/sagemath:10.8 /bin/bash
   • good - provides a clean shell from which Sage can be started explicitly
   • bad - changes the user experience
   • bad - python3 is not directly available; users must use sage -python3 (and similarly sage -pip)
3. Remove the export PATH=... line from ~/.bashrc
   • good- this would resolve the issue without changing user-facing commands - docker run -it sagemath/sagemath:latest /bin/bash would behave as expected
   • bad - this may reintroduce the issue that was originally addressed by
     Fix broken docker/Dockerfile #38049; at the moment, that PR is difficult for me to fully interpret, and I am not confident about the potential side effects.

I would appreciate guidance on which direction would be preferred

[hannob]

I just got hit by this and was very confused why I couldn't execute python within the sage container.

Looking at this, I believe that the code in the Dockerfile is probably not doing what was intended here. From https://github.com/sagemath/sage/blob/develop/docker/Dockerfile#L235C1-L235C63 we have:

RUN echo "export PATH=$SAGE_ROOT/local/bin:$PATH" >> ~/.bashrc

I believe the intention here is to have export PATH=$SAGE_ROOT/local/bin:$PATH ending up in ~/.bashrc. But that's not what's happening. It is resolving the variables $SAGE_ROOT and $PATH that are present in the build environment, so we end up with export PATH=/home/sage/sage/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin there.

I think the solution is simply to escape the $ character or use single quotes that avoid resolving those variables.

[real-or-random]

I think the solution is simply to escape the $ character or use single quotes that avoid resolving those variables.

Yes, but only for $PATH. If I understand correctly, $SAGE_ROOT is supposed to be taken from the build environment.

[xbonnetain]

With the merge of #42139 I see an update on the (still faulty) behaviour:

$ docker run -it sagemath/sagemath:latest bash
sage@567b96c93f5b:~$ sage
Traceback (most recent call last):
  File "/home/sage/sage/src/bin/sage-ipython", line 9, in <module>
    from sage.misc.banner import banner
ModuleNotFoundError: No module named 'sage'

[soehms]

With the merge of #42139 I see an update on the (still faulty) behaviour:

To be precise: It hasn't been merged yet, but the patched image has already been pushed to DockerHub.