This repo contains the Docker images for the Fornax Platform deployments. It produces the jupyterhub computing environments.
The following is a general description of the images:
-
Each image is in its own directory (e.g.
fornax-base
andfornax-main
). -
jupyter-base
is a custom jupyterlab image that matches the base-notebook. -
fornax-base
is a base image for other images in fornax. Other images should use it as a starting point. -
Jupyterlab is installed in the base conda environment. Basic astronomy and analysis packages are installed in a environment called
notebook
, which is the default environment for running analysis. -
Each demo notebook has its own environment. These environments are created during the build using the
requirements.txt
files associated with individual notebooks (managed withuv
). Notebook environment have names:py-{notebook-name}
. -
The
scripts/build.py
script is used when building the images. The script has many options. Runpython scripts/build.py -h
for detailed help. Example runs include:- To build an image locally:
python scripts/build.py image-name
. Whereimage-name
isfornax-base
,fornax-main
etc. The image will be tagged asghcr.io/nasa-fornax/fornax-images/{image-name}:{branch-name}
- Adding
--push
pushes the images the the github container registry. - Adding
--ecr-endpoint $ENDPOINT --trigger-ecr
notify$ENDPOINT
that a new image has been built. This is used as part of the build CI.
- To build an image locally:
-
Building an image that starts from
fornx-base
will trigger theONBUILD
sections defined infornax-base/Dockerfile
, which include:- If
apt.txt
exits, it will be parsed for the list of the system software to be installed withapt-get
. - If
build-*
files exist, the scripts are run during the build. - If
conda-{env}.yml
exists, it is used to create a conda environment called{env}
. - Additionally, if
conda-{env}-lock.yml
exists, it locks the versions of the installed libraries. To create this-lock
file, or updated it, pass--update-lock
to the build scriptscripts/build.py
. This will first create or update the conda environment from theconda-{env}.yml
file, then generate a newconda-{env}-lock.yml
from the installed packages. - If
requirements-{env}.txt
exists, it is used to create a virtual python environment{env}
(usinguv venv
). Experience showed that having packages managed withpip
rather thanconda
, reduces the chance of conflicts. Conda-managed environments can create the undesired situations (e.g. #20) where conda install a version of a package and pip install another one. - If
introduction.md
file exists in an image folder, it is converted to html withpandoc
and copied to/opt/scritps
in the image. During session startup, thepre-notebook.sh
script makes a copy in the user's~/notebooks
. This can be served as a landing page by settingJUPYTERHUB_DEFAULT_URL
the jupyterhub deployment code.
- If
-
jupyter-base
: is a custom jupyterlab image that matches the base-notebook in the jupyter stack. It is build here to allow for customization, including the possibility of using conda alternatives, such as pixi (in the future). -
fornax-base
: is the base image that all other fornax images start from. It contains jupyter and the basic science tools needed for deployment in the fornax project. -
fornax-main
: Main Fornax image that was used for the demo notebooks. It contains several notebook kernels. Thenotebook
environment contains general astronomy and science tools. Additionally, each demo notebook has its own environment (or kernel). -
fornax-hea
: Fornax image containign high energy software. For now, it has heasoft as installed as a conda package.