docker-compose.ci.yml: stale build-baked /app/*.py silently shadows live-mounted /app/backend/*.py

[johanzander]

Background

Discovered while running local runtime verification for #219/PR #224. docker-compose.ci.yml is used for local dev/CI verification with live-editable source, but backend-root Python files (app.py, api_conversion.py, api.py, etc. — anything directly under backend/) silently do not live-reload, even though core/bess/* and backend/tests/* do.

The problem

backend/Dockerfile.dev does:

WORKDIR /app
COPY backend/ /app/         # flat copy — backend/app.py -> /app/app.py
COPY core/bess /app/core/bess
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8080"]

docker-compose.ci.yml then bind-mounts the same source on top, but at a different path:

volumes:
  - ./backend:/app/backend:ro   # -> /app/backend/app.py (separate from /app/app.py)
  - ./core:/app/core:ro
environment:
  - PYTHONPATH=/app/backend:/app

This creates two copies of every backend-root module: a stale one baked into the image at /app/*.py (from the COPY step, frozen at build time) and a live one at /app/backend/*.py (from the bind mount). core/bess/* doesn't have this problem — it's only ever /app/core/bess/, no duplicate.

Because uvicorn is invoked as uvicorn app:app from WORKDIR /app, Python inserts '' (resolving to cwd, /app) at sys.path[0] — before the PYTHONPATH entries (/app/backend, /app). So import app, import api_conversion, import api, etc. all resolve to the stale, build-time-baked copy at /app/*.py, not the live-mounted /app/backend/*.py.

Impact

A plain podman restart / docker compose restart (or stop+start) after editing any backend-root file does not pick up the edit — the container keeps running the code from the last --build. This is silent: no error, no warning, the container just serves stale behavior. It cost significant time during #219's verification (a deliberately-reverted line to test a crash scenario appeared to have no effect, because the running process was never executing the edited file — confirmed by direct python3 -c "import app" with cwd=/app/backend, which correctly picks up live edits since that puts /app/backend ahead of /app in sys.path).

Only core/bess/* edits and backend/tests/* edits (no baked duplicate) are actually live. Anyone relying on "edit + restart container" for app.py/api.py/api_conversion.py/etc. during E2E work is silently testing old code.

Proposed fix

One of:

1. Stop baking backend-root files into the image at all in Dockerfile.dev (rely entirely on the bind mount + PYTHONPATH), or
2. Set WORKDIR /app/backend (or run cd /app/backend && uvicorn app:app ...) so sys.path[0] resolves to the live-mounted directory first, or
3. Drop the /app/backend bind mount and PYTHONPATH entry entirely and just bind-mount straight onto /app/*.py (matching how core/bess already works — mount ./backend:/app:ro... though this would need care since /app also holds installed dependencies' entrypoint expectations).

Whichever fix, add a one-line comment in docker-compose.ci.yml explaining why the chosen layout is safe, so this doesn't regress.

Files affected

• backend/Dockerfile.dev
• docker-compose.ci.yml

[bess-manager-claude-bot]

Thanks for the detailed report. This is a Docker dev-environment issue where backend-root modules baked into the image at build time shadow the live-mounted copies, causing edits to app.py, api.py, etc. to go unnoticed until a full rebuild. You've clearly identified the root cause and three viable fixes. This is queued for implementation review. Reply @claude-bot analyze to start deep code-level analysis.