Skip to content

Commit 4bf2f5e

Browse files
chad-earthscopechad-earthscope
authored
Merge pull request #56 from sparafina-earthscope/geolab-slim
GeoLab slim
2 parents 039387c + 710fe3b commit 4bf2f5e

16 files changed

Lines changed: 1238 additions & 444 deletions

‎geolab-slim/Dockerfile‎

Lines changed: 24 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -1,62 +1,25 @@
11
# syntax=docker/dockerfile:1
2-
3-
# ── Build stage: compile pyrocko and install conda env ───────────────────────
4-
FROM quay.io/jupyter/base-notebook:latest AS builder
5-
6-
USER root
7-
8-
RUN apt-get update --quiet && \
9-
apt-get install --quiet --yes --no-install-recommends \
10-
git \
11-
python3-dev \
12-
build-essential \
13-
gcc \
14-
gfortran \
15-
g++ \
16-
make && \
17-
apt-get clean && \
18-
rm -rf /var/lib/apt/lists/*
19-
20-
USER ${NB_UID}
21-
22-
RUN git clone https://git.pyrocko.org/pyrocko/pyrocko.git /tmp/pyrocko && \
23-
cd /tmp/pyrocko && \
24-
python install.py deps conda --yes && \
25-
python install.py user --y && \
26-
rm -rf /tmp/pyrocko
27-
28-
COPY --chown=${NB_UID}:${NB_GID} environment.yaml /tmp/environment.yaml
29-
30-
RUN mamba env update --name base --file /tmp/environment.yaml && \
31-
pip install gnss-lib-py --ignore-requires-python && \
32-
mamba clean --all --yes && \
33-
find /opt/conda -name '__pycache__' -type d -exec rm -rf {} + 2>/dev/null || true && \
34-
find /opt/conda -name '*.pyc' -delete 2>/dev/null || true
35-
36-
# ── Runtime stage ─────────────────────────────────────────────────────────────
37-
FROM quay.io/jupyter/base-notebook:latest
38-
39-
LABEL maintainer="your-name <your@email.com>"
40-
LABEL description="Geoscience JupyterLab environment"
41-
42-
USER root
43-
44-
RUN apt-get update --quiet && \
45-
apt-get install --quiet --yes --no-install-recommends \
46-
git \
47-
gmt-dcw gmt-gshhg \
48-
libgfortran5 \
49-
libgomp1 \
50-
ftp && \
51-
apt-get clean && \
52-
rm -rf /var/lib/apt/lists/* && \
53-
rm -rf /home/joyvan/work
54-
55-
COPY --from=builder /opt/conda /opt/conda
56-
COPY --from=builder --chown=${NB_UID}:${NB_GID} /home/jovyan /home/jovyan
57-
COPY --chown=${NB_UID}:${NB_GID} test_suite.py /etc/tests/test_suite.py
58-
59-
USER ${NB_UID}
60-
61-
ENV JUPYTER_ENABLE_LAB=yes
62-
CMD ["start-notebook.py"]
2+
# ──────────────────────────────────────────────────────────────
3+
# geolab-slim — built on pangeo/base-image.
4+
#
5+
# Files picked up automatically by the parent's ONBUILD machinery:
6+
# environment.yml → conda env "notebook" installed from this
7+
# conda-lock.yml → (optional, preferred) no-solve install
8+
# start → entrypoint script (becomes /srv/start)
9+
#
10+
# Build:
11+
# docker build --platform linux/amd64 -t geolab-slim .
12+
# ──────────────────────────────────────────────────────────────
13+
FROM pangeo/base-image:latest
14+
15+
LABEL org.opencontainers.image.title="geolab-slim" \
16+
org.opencontainers.image.authors="sophia.parafina@earthscope.org"
17+
18+
# PROJ/GDAL resource directories.
19+
ENV PROJ_DATA=/srv/conda/envs/notebook/share/proj \
20+
PROJ_LIB=/srv/conda/envs/notebook/share/proj \
21+
GDAL_DATA=/srv/conda/envs/notebook/share/gdal
22+
23+
# Default command for standalone use. JupyterHub spawning passes
24+
# its own command (jupyterhub-singleuser), which start respects.
25+
CMD ["jupyter", "lab", "--ip=0.0.0.0", "--no-browser"]

‎geolab-slim/README.md‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
# Building the GeoLab "slim" image
2+
3+
The GeoLab slim image is a base image containing minimal software for working in the platform.
4+
5+
GeoLab environments run as **containers** based on **images** — self-contained packages that include an operating system, software libraries, and Python packages together. This guide documents the steps using Docker to create the geolab-slim image.
6+
7+
GeoLab images use the `pangeo/base-notebook` image as the starting point. This base image contains the fundamentals needed for an environment in JupyterHub. You customize it by editing four files before building:
8+
9+
| File | What it controls |
10+
| :--- | :--------------- |
11+
| `apt.txt` | System-level software (installed via `apt`) |
12+
| `environment.yml` | Conda packages and channels |
13+
| `requirements.txt` | Python packages from PyPI (installed via `pip`) |
14+
| `start` | The command that launches when the container starts |
15+
16+
---
17+
18+
## Installing System Software with apt
19+
20+
`apt` is the Ubuntu package manager — it installs system-level tools like compilers, runtime libraries, and command-line utilities. Add any packages you need, one per line, to `apt.txt`.
21+
22+
> **Tip:** Only add packages here that aren't available through conda. Most scientific Python libraries are better managed in `environment.yml`.
23+
24+
---
25+
26+
## Installing Conda Packages
27+
28+
Conda manages Python (and non-Python) packages within isolated environments. Edit `environment.yml` to add packages by name under the appropriate section. Always use the `conda-forge` channel for the broadest package availability.
29+
30+
> **Tip:** Prefer conda packages over pip when a package is available in both. Conda resolves environment-wide dependencies more reliably.
31+
32+
---
33+
34+
## Installing pip Packages
35+
36+
Some packages are only available on PyPI (Python's package index) and must be installed with `pip`. Add them to `requirements.txt`, one per line. You can pin a specific version with `==` to ensure reproducibility.
37+
38+
> **Tip:** Pin versions for packages critical to your workflow (e.g., `earthscope-sdk==1.4.1`). This prevents silent breakage when upstream packages release updates.
39+
40+
---
41+
42+
## Start Script
43+
44+
`start` runs when a user launches a container. Its job is to start the main process — typically JupyterLab.
45+
For this image the contents simply executes the command provided by the Hub.
46+
47+
---
48+
49+
## Building and Pushing the Image
50+
51+
Once your configuration files are ready, you build the image locally and push it to a container registry so GeoLab can access it.
52+
53+
**Step 1 — Build the image**
54+
55+
The `--platform linux/amd64` flag ensures the image runs on standard cloud hardware regardless of whether you're building on an Apple Silicon or an Intel machine. `Dockerfile` is the GeoLab-specific Dockerfile that wires together your four config files.
56+
57+
```bash
58+
docker build --no-cache -f Dockerfile \
59+
--platform linux/amd64 \
60+
-t username/geolab-slim:latest-amd64 .
61+
```
62+
63+
Replace `username` with your Docker Hub username (or your registry path) and `latest-amd64` with your version tag of choice.
64+
65+
> **What does `--no-cache` do?** It forces Docker to re-run every build step from scratch, ensuring your latest `apt.txt`, `environment.yml`, and `requirements.txt` changes are picked up rather than reused from a previous build.
66+
67+
**Step 2 — Push to a registry**
68+
69+
Push the finished image to Docker Hub, AWS ECR, or another registry so GeoLab can pull it:
70+
71+
```bash
72+
docker push username/geolab-slim:latest-amd64
73+
```
74+
75+
> **First time?** You'll need to log in first with `docker login` (Docker Hub) or the appropriate CLI for your registry.
76+
77+
---
78+
79+
## Running Your Image in GeoLab
80+
81+
1. Open GeoLab.
82+
2. Choose **Environment → Other**.
83+
84+
![Choose Environment](./images/choose_environment.png)
85+
86+
3. Enter the full image name from your registry, e.g.:
87+
```
88+
username/geolab-slim:latest-amd64
89+
```
90+
91+
![Enter image name](./images/custom_image.png)
92+
93+
4. Select **Start**.
94+
95+
GeoLab will pull and launch your custom environment. The first launch may take a minute while the image downloads.
96+
97+
## Final steps
98+
99+
If the image works as desired create a PR against the `main` branch for review by the ownership team.
100+
Once accepted the production image will be built and deployed automatically in the EarthScope AWS container registry.
101+
102+
## Local build and test
103+
104+
An image built for your machine can be created using:
105+
106+
```bash
107+
docker build --no-cache -f Dockerfile -t geolab-slim:test .
108+
```
109+
110+
> **Not for GeoLab!** Without the guarantee of `--platform linux/amd64` this image is not for GeoLab
111+
112+
Run a container using this image with:
113+
114+
```bash
115+
docker run --rm -p 8888:8888 geolab-slim:test
116+
```
117+
118+
Then use a web browser to connect to: `http://127.0.0.1:8888/lab`,
119+
open `smoke_tests.ipynb` and run all cells.

‎geolab-slim/apt.txt‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
git
2+
build-essential
3+
gfortran
4+
make
5+
gmt-dcw
6+
gmt-gshhg

geolab-slim/environment.yaml renamed to geolab-slim/environment.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,11 +48,11 @@ dependencies:
4848
- pygmt
4949
- pip
5050
- pip:
51-
- earthscope-sdk[arrow]==1.4.0
51+
- earthscope-sdk[arrow]==1.4.1
5252
- earthscope-cli==1.2.0
5353
- earthscopestraintools
5454
- gnssrefl
5555
- hypoinvpy
5656
- jupyterlab-jupyterbook_navigation
5757
- pynlloc
58-
- pyocto
58+
- pyocto

‎geolab-slim/images/choose_environment.png‎

50.8 KB
Loading

‎geolab-slim/images/custom_image.png‎

54.1 KB
Loading

‎geolab-slim/previous_iterations/Dockerfile-multi-stage‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
# syntax=docker/dockerfile:1
2+
3+
# ── Build stage: compile pyrocko and install conda env ───────────────────────
4+
FROM --platform=$BUILDPLATFORM quay.io/jupyter/base-notebook:latest AS builder
5+
6+
USER root
7+
8+
ENV DEBIAN_FRONTEND=noninteractive
9+
10+
RUN apt-get update --quiet && \
11+
apt-get install --quiet --yes --no-install-recommends \
12+
git \
13+
python3-dev \
14+
build-essential \
15+
gcc \
16+
gfortran \
17+
g++ \
18+
make && \
19+
apt-get clean && \
20+
rm -rf /var/lib/apt/lists/* && \
21+
conda install -y -c conda-forge zstandard && \
22+
conda clean -afy
23+
24+
USER ${NB_UID}
25+
26+
# RUN git clone https://git.pyrocko.org/pyrocko/pyrocko.git /tmp/pyrocko && \
27+
# cd /tmp/pyrocko && \
28+
# python install.py deps conda --yes && \
29+
# python install.py user --y && \
30+
# rm -rf /tmp/pyrocko
31+
32+
COPY --chown=${NB_UID}:${NB_GID} environment.yaml /tmp/environment.yaml
33+
34+
# RUN conda update --name notebook --file /tmp/environment.yaml && \
35+
# # conda install -y -c conda-forge zstandard && \
36+
# pip install gnss-lib-py --ignore-requires-python && \
37+
# conda clean --all --yes && \
38+
# find /opt/conda -name '__pycache__' -type d -exec rm -rf {} + 2>/dev/null || true && \
39+
# find /opt/conda -name '*.pyc' -delete 2>/dev/null || true
40+
41+
# Create a fresh named environment so conda respects python=3.13.13,
42+
# then register it as a Jupyter kernel and clean up.
43+
RUN conda env create --name notebook --file /tmp/environment.yaml && \
44+
/opt/conda/envs/notebook/bin/pip install gnss-lib-py --ignore-requires-python && \
45+
/opt/conda/envs/notebook/bin/python -m ipykernel install \
46+
--prefix /opt/conda \
47+
--name notebook \
48+
--display-name "Python (notebook)" && \
49+
conda clean --all --yes && \
50+
find /opt/conda -name '__pycache__' -type d -exec rm -rf {} + 2>/dev/null || true && \
51+
find /opt/conda -name '*.pyc' -delete 2>/dev/null || true
52+
53+
54+
# ── Runtime stage ─────────────────────────────────────────────────────────────
55+
FROM quay.io/jupyter/scipy-notebook:latest
56+
57+
LABEL maintainer="earthscope <geolab@earthscope.org>"
58+
LABEL description="GeoLab JupyterLab environment"
59+
60+
USER root
61+
62+
RUN apt-get update --quiet && \
63+
apt-get install --quiet --yes --no-install-recommends \
64+
gmt-dcw gmt-gshhg \
65+
libgfortran5 \
66+
libgomp1 \
67+
ftp && \
68+
apt-get clean && \
69+
rm -rf /var/lib/apt/lists/* && \
70+
rm -rf /home/joyvan/work
71+
72+
COPY --from=builder /opt/conda /opt/conda
73+
COPY --from=builder --chown=${NB_UID}:${NB_GID} /home/jovyan /home/jovyan
74+
COPY --chown=${NB_UID}:${NB_GID} test_suite.py /etc/tests/test_suite.py
75+
76+
USER ${NB_UID}
77+
78+
ENV JUPYTER_ENABLE_LAB=yes
79+
CMD ["start-notebook.py"]

‎geolab-slim/previous_iterations/Dockerfile-single-stage‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# syntax=docker/dockerfile:1
2+
3+
FROM quay.io/jupyter/base-notebook:latest
4+
# FROM pangeo/base-notebook:d0ea2af
5+
6+
LABEL maintainer="earthscope <geolab@earthscope.org>"
7+
LABEL description="GeoLab JupyterLab environment"
8+
9+
USER root
10+
11+
RUN apt-get update --quiet && \
12+
apt-get install --quiet --yes --no-install-recommends \
13+
git \
14+
gmt-dcw \
15+
gmt-gshhg \
16+
libgfortran5 \
17+
libgomp1 && \
18+
apt-get clean && \
19+
rm -rf /var/lib/apt/lists/*
20+
21+
COPY environment.yaml /tmp/environment.yaml
22+
23+
# Create a fresh conda environment named 'notebook' with Python 3.13.13.
24+
# conda env create (not update) on a named env always honours the python= pin.
25+
# --force removes any pre-existing 'notebook' env from the base image.
26+
# python --version after create confirms the correct interpreter was installed.
27+
# Initialise conda for bash so 'conda activate' works in terminal sessions.
28+
29+
RUN mamba env create --file /tmp/environment.yaml
30+
RUN /opt/conda/envs/geolab/bin/python -m ipykernel install \
31+
--prefix /opt/conda \
32+
--name geolab \
33+
--display-name "Python (geolab)" && \
34+
conda clean --all --yes
35+
36+
37+
# Prepend the notebook env so it is the default Python in the container.
38+
ENV PATH="/opt/conda/envs/geolab/bin:${PATH}"
39+
40+
# Switch to jovyan so conda init writes to /home/jovyan/.bashrc, not /root/.bashrc.
41+
USER ${NB_UID}
42+
RUN conda init bash

0 commit comments

Comments
 (0)