Bake app code into image; pin dev to :main, prod by digest (Fixes #169) (#171)

* build: add .dockerignore to keep node_modules/.venv/.git out of image context

* build: bake application code into image (COPY . /app); drop runtime git clone dependency

* ci: build image on push to main + release tags; tag :main/:sha/:vX.Y.Z; cron refreshes deps via no-cache

* docs: rewrite rollout workflow for baked-image model (dev :main, prod digest-pinned)

* build: fix .dockerignore excluding the tiles/ runtime package; trim non-runtime dirs; clarify cron cache comment

* docs: add baked-images deployment-pinning implementation plan

---------

Co-authored-by: Carl Boettiger <cboettig@berkeley.edu>

Author: boettiger-lab-llm-agent[bot]

.dockerignore

@@ -0,0 +1,17 @@
+.git
+.github
+.venv
+node_modules
+**/__pycache__
+*.pyc
+.pytest_cache
+.worktrees
+.claude
+.roo
+.continue
+.vscode
+benchmarks
+docs
+examples
+memory
+tests

.github/workflows/docker.yml

@@ -3,9 +3,7 @@ name: Build Docker image
 on:
   push:
     branches: [main]
-    paths:
-      - Dockerfile
-      - .github/workflows/docker.yml
+    tags: ['v*']
   schedule:
     - cron: '0 6 * * 1'  # Mondays 06:00 UTC — refresh base image + unpinned deps
   workflow_dispatch:
@@ -26,24 +24,57 @@ jobs:
           username: ${{ github.actor }}
           password: ${{ secrets.GITHUB_TOKEN }}
 
+      - name: Set up Buildx
+        uses: docker/setup-buildx-action@v3
+
       - name: Compute image tags
         id: tags
         run: |
-          # Always tag :latest. Only tag :<sha> on push events so scheduled
-          # rebuilds don't mutate an immutable content-addressed tag.
-          tags="ghcr.io/boettiger-lab/mcp-data-server:latest"
-          if [ "${{ github.event_name }}" = "push" ]; then
-            tags="$tags"$'\n'"ghcr.io/boettiger-lab/mcp-data-server:${{ github.sha }}"
-          fi
+          repo=ghcr.io/boettiger-lab/mcp-data-server
+          case "${{ github.event_name }}" in
+            push)
+              if [ "${{ github.ref_type }}" = "tag" ]; then
+                # Release tag (vX.Y.Z): immutable version tag + immutable sha.
+                tags="$repo:${{ github.ref_name }}"$'\n'"$repo:${{ github.sha }}"
+              else
+                # Push to main: moving dev tag + immutable sha.
+                tags="$repo:main"$'\n'"$repo:${{ github.sha }}"
+              fi
+              ;;
+            *)
+              # schedule / workflow_dispatch: dep+base refresh. Move ONLY the dev
+              # tag — never mint a :sha or :vX.Y.Z from a non-push event.
+              tags="$repo:main"
+              ;;
+          esac
           {
             echo "tags<<EOF"
             echo "$tags"
             echo "EOF"
           } >> "$GITHUB_OUTPUT"
 
       - name: Build and push
+        id: build
         uses: docker/build-push-action@v6
         with:
           context: .
           push: true
           tags: ${{ steps.tags.outputs.tags }}
+          # Pushes reuse the cached deps layer (fast code-only builds). The weekly
+          # cron sets no-cache + pull so it re-resolves unpinned deps and pulls a
+          # fresh base image (security patches); cache-to still warms the cache.
+          # (When no-cache=true, cache-from is ignored by design; cache-to is kept
+          # so the next code-push still gets a warm cache.)
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
+          no-cache: ${{ github.event_name == 'schedule' }}
+          pull: ${{ github.event_name == 'schedule' }}
+
+      - name: Report image digest
+        run: |
+          {
+            echo "### Image digest"
+            echo '```'
+            echo "${{ steps.build.outputs.digest }}"
+            echo '```'
+          } >> "$GITHUB_STEP_SUMMARY"

AGENTS.md

@@ -50,26 +50,44 @@ This is intentional and load-bearing. Several things depend on it:
 
 ### Rollout workflow
 
-**Merge to `main` → redeploy dev only:**
+Application code is **baked into the image** (`COPY . /app` in the `Dockerfile`); pods no
+longer `git clone` at startup. `docker.yml` builds on every push to `main` and on `vX.Y.Z`
+release tags. The image is the unit of release.
+
+**Tags CI produces:**
+- `:main` — moving; rebuilt on every push to `main` and by the weekly cron. **dev** tracks this.
+- `:<git-sha>` — immutable; one per commit.
+- `:vX.Y.Z` — immutable; built on release tags. **prod** pins this (by digest, below).
+
+**Merge to `main` → redeploy dev:**
 ```
 kubectl apply -f k8s/dev-deployment.yaml
 kubectl rollout restart deployment/dev-duckdb-mcp -n biodiversity
 ```
-
-**Tag a release → redeploy prod:**
+dev pins `:main` with `imagePullPolicy: Always`, so the restart pulls the freshest build.
+**Wait for the `docker.yml` run on your merge to go green first** — rolling before the
+image is pushed gives `ImagePullBackOff`.
+
+**Tag a release → redeploy prod (promote by digest):**
+1. `git tag vX.Y.Z && git push origin vX.Y.Z`, then wait for `docker.yml` to build `:vX.Y.Z`.
+2. Read the digest from the build run's job summary, or:
+   `docker buildx imagetools inspect ghcr.io/boettiger-lab/mcp-data-server:vX.Y.Z --format '{{.Manifest.Digest}}'`
+3. Set `image: ghcr.io/boettiger-lab/mcp-data-server:vX.Y.Z@sha256:<digest>` in
+   `k8s/deployment.yaml` (separate commit).
+4. `kubectl apply -f k8s/deployment.yaml`
+5. `kubectl rollout restart deployment/duckdb-mcp -n biodiversity`
+
+prod pins an immutable `:vX.Y.Z@sha256:…` (tag for humans, digest enforced — if they ever
+disagree, the digest wins). **Never apply prod while the manifest points at an image CI
+hasn't built yet** — the rollout stalls on `ImagePullBackOff`. `kubectl apply` must precede
+`rollout restart`; a git push alone does not update the cluster.
+
+Verify all prod replicas converge on a single digest after rollout:
 ```
-kubectl apply -f k8s/deployment.yaml
-kubectl rollout restart deployment/duckdb-mcp -n biodiversity
+kubectl -n biodiversity get pods -l app=duckdb-mcp \
+  -o custom-columns='NAME:.metadata.name,IMAGE:.status.containerStatuses[0].imageID'
 ```
 
-Prod clones a pinned tag, so a release is two steps: **first** bump the `--branch vX.Y.Z`
-pin in `k8s/deployment.yaml` to the new tag (separate commit, as in #151), **then** apply.
-Never `kubectl apply -f k8s/deployment.yaml` to prod while the pin still points at the
-previous tag — `deployment.yaml` changes (e.g. new `/healthz` probes) can reference code
-the pinned tag doesn't yet have, and the rollout will stall on failing probes.
-
-`kubectl apply` must precede `rollout restart` — a git push alone does not update the cluster.
-
 ---
 
 This project uses two distinct, asynchronous AI agent processes. Do not confuse them.

Dockerfile

@@ -1,21 +1,25 @@
 FROM python:3.12-slim
 
+# ca-certificates for HTTPS to S3 / STAC. git is intentionally NOT installed:
+# code is baked into the image (COPY below), not cloned at pod startup.
 RUN apt-get update \
-    && apt-get install -y --no-install-recommends git ca-certificates \
+    && apt-get install -y --no-install-recommends ca-certificates \
     && rm -rf /var/lib/apt/lists/*
 
-RUN pip install --no-cache-dir \
-        mcp \
-        duckdb \
-        pandas \
-        uvicorn \
-        tabulate \
-        pystac \
-        requests
+WORKDIR /app
+
+# Dependency layer — re-resolves ONLY when requirements.txt changes, so code-only
+# rebuilds reuse it (with GHA cache, those finish in seconds). The weekly cron
+# builds with --no-cache so this layer actually re-resolves the unpinned deps.
+COPY requirements.txt .
+RUN pip install --no-cache-dir -r requirements.txt
 
 # Pre-install DuckDB extensions so pods start offline-capable.
 # Uses the default extension directory (~/.duckdb/extensions/) which the runtime
 # will find automatically since the container runs as root.
 RUN python -c "import duckdb; c = duckdb.connect(); c.sql('INSTALL httpfs; INSTALL spatial; INSTALL h3 FROM community')"
 
-WORKDIR /app
+# Application code last, so a code change rebuilds only this cheap layer.
+COPY . /app
+
+CMD ["python", "server.py"]