Revisit root permissions and entrypoint for user management

[minrk]

In start.sh, we support setting up the user in a variety of ways that currently require the container to be started as root. #552, #553, #559 introduce a mechanism to handle a subset of those cases when the user is in the root group, as is typical with openshift. #553 initially did so in an ENTRYPOINT instead of CMD in order to benefit even when CMD is overridden, e.g. directly to jupyterhub-singleuser in kubespawner. That logic now resides in start.sh with everything else (#559), but perhaps much of start.sh ought to move to an ENTRYPOINT instead of the CMD.

Things to consider:

1. how much can/should we accomplish with the root group instead of the root user? I'd love it if we could stop requiring folks to launch the image as the root user, since a slip-up there can be a major permission issue
2. how much of the permissions/user setup belongs in an ENTRYPOINT rather than start.sh? Pro of ENTRYPOINT is that overriding CMD preserves permissions/user-management logic. Con of ENTRYPOINT is that opting out of user-management behavior is a bit harder if desired. So if we move things to the entrypoint, we should make sure that default behavior is always safe and almost always desirable.

Current capabilities in start.sh that require root permissions:

• update $HOME to /home/$NB_USER if username is set specifically
• rename jovyan user to $NB_USER if set
• set UID/GID of NB_USER
• grant $NB_USER sudo permissions
• chown $HOME (typically just the empty-directory volume case)

[parente]

I've stalled on creating the ReadTheDocs site, partially because I'm at the section of the doc where I have to document all the various options the images support. It's daunting.

So I was planning to open an issue this morning to discuss what options we currently support (e.g., NB_UID), why they were originally needed, and if they're still needed given the usage patterns that have emerged.

Then I saw this new ticket about root permissions (thanks @minrk !) and wonder if we should tackle the two topics together:

• Can we shed any of the options we currently support?
• How much do we really need to do in entrypoint / cmd that requires root?

[aneagoe]

I'm currently using https://github.com/GrahamDumpleton/openshift3-jupyter-stacks to be able to deploy on OpenShift. However, since the dockerfile there specifies changes to /home/$NB_USER, it results in an additional ~3GB layer to the base image, just because it's changing permissions.

I can imagine if this enhancement is resolved, then these images would run natively on OpenShift.
The biggest challenge in doing this seems to be the requirement to run as root since some of the current capabilities require it. However, there's not much clarity as per why those features are required in the first place.

Can someone shed some light into why it's required to be able to run as a different user?

[parente]

@GrahamDumpleton made quite a few contributions to this repository lately, including documentation about using these images on OpenShift 3 (https://github.com/jupyter/docker-stacks/tree/master/examples/openshift). Perhaps the problem has been mitigated already?

[GrahamDumpleton]

The docker-stacks images don't have all the changes I would like to see, but based on reactions have got, I doubt that will ever be able to get the last change I want done that would allow the images to run on OpenShift out of the box without any special configuration. That one change I would like to still have made is having NB_GID be 0 and not that of users. So by default notebook user would run with gid of 0 and all directories and files it needs to write to being gid of 0 as well.

So have done as much as I can. They will run, but you need to configure OpenShift, the Kubernetes environment with multi user RBAC setup, or any other more secure Docker environment, to run the containers with supplemental group of 100, being that for users.

As pointed out, see the examples directory. For Source-to-Image builds there is also:

• https://github.com/jupyter/docker-stacks/tree/master/examples/source-to-image

I am gradually pulling all information together about running Jupyter on OpenShift at:

• https://www.jupyteronopenshift.org/

It is a self signed certificate on the site for now, but that will be fixed.

There is also lots of other repos and examples related to running JupyterHub, kernel gateways, nbviewer, native OpenShift S2I based notebooks etc., at:

• https://github.com/jupyter-on-openshift

[parente]

That one change I would like to still have made is having NB_GID be 0 and not that of users. So by default notebook user would run with gid of 0 and all directories and files it needs to write to being gid of 0 as well.

Maybe my memory is faulty, but I remember some questions about whether using the root group could lead to security issues and some answers back that it looked OK to OpenShift folks. After that, I think the conversation trailed off and was lost when folks, myself included, suggested that changes be made one PR at a time.

I'm open to switching the default user gid to 0 as long as it doesn't regress other features of the images. I think @minrk is too from his comments at the start of this issue, albeit for the separate reason of trying to mitigate the need to launch containers as uid=0 for certain features.

[GrahamDumpleton]

If these images were only ever used as is there would be no issue changing. The problem is the effect on people who are creating derived images. You can’t assume that people when adding stuff in derived images are setting user/group based on NB_UID/NB_GID environment variables, if they are at least trying to match permissions. So it isn’t a security issue, just that derived images people are creating could be broken by changing the group to GID of 0.

[parente]

Sorry, I should have been clearer. By "regress other features" I meant "break without the ability to fix".

Documenting best practices for deriving images from the core definitions here, especially the base and minimal images, is fodder for the ReadTheDocs site slowly being worked in #517 and associated PRs. If we document examples of how to properly set permissions in derived images (e.g., fix-permissions script) and we tag the last image to use the gid=100 setting for users that aren't ready to make the switch quite yet, I think breaking backward compatibility is fair game.

[aneagoe]

@GrahamDumpleton thanks a lot for all your efforts with regards to getting jupyter on OpenShift, it was a huge help.
It seems to be already better than I thought, I didn't know that just applying supplemental group 100 would work with existing upstream images. That makes it easier. Still, it would be ideal to run with gid=0 straight off.

[minrk]

My main concern about defaulting to GID=0 is that it seems to be the wrong thing to do for the much more common case of people running these docker images outside of kubernetes/openshift-style locked-down environments. See this thread where running users as gid=0 with the current state of things makes it trivial to become the root user. Only in extra-secure contexts like openshift does it seem to be a safe and correct choice use run with gid=0. Since that's not the primary target case for these docker stacks, I'm not sure there's a good way to serve both.

I also want it to be clear that these stacks are far from the only way to install Jupyter, and we shouldn't be trying to make these stacks serve all use cases. I think the current level of configurability is a source of significant complications.

I suspect for cases like openshift with specific demands, using something like s2i to build images with Jupyter installed via conda and/or pip with specific choices for user/group permissions may well be the preferable, simpler solution to trying to make these stacks work optimally in all possible environments.

[GrahamDumpleton]

Arrgggh. Totally forgot that issue was raised around use of su. With that PR closed and work moved to another, that probably didn't get discussed further as well as it should have at the time.

[GrahamDumpleton]

Not sure yet what issues this may cause with existing functionality in the image which uses su and sudo, but you could restrict use of su to users in the wheel group.

RUN echo "auth           required        pam_wheel.so use_uid" >> /etc/pam.d/su

Result then is:

$ docker run -u 100000 --rm -it sutest
(app-root)id
uid=100000 gid=0(root) groups=0(root)
(app-root)cp /etc/passwd /tmp/passwd
(app-root)rootpass=`openssl passwd -1 root`
(app-root)cat /tmp/passwd | sed "s/root:x/root:${rootpass}/" > /etc/passwd
(app-root)
(app-root)su root
Password:
su: Error in service module
(app-root)id
uid=100000 gid=0(root) groups=0(root)

[GrahamDumpleton]

You would though have to also rollback the change made to make /etc/group writable as if left that way someone could add themselves to group wheel anyway.

The addition of a corresponding entry to /etc/group for an assigned user ID not in /etc/passwd was not essential and I don't know of a specific case where lack of it would cause failure of Jupyter notebooks or other Python packages.