image build improvements

[claire-peters]

Summary

This PR contains several improvements to the Docker image build process and dependency updates.

Dockerfile changes

• Fix apt-get invocation — split the double-apt-get install into a single, properly structured RUN layer with one package per line. Previously the first apt-get install -y --no-install-recommends was a no-op (no packages listed), followed by a second install that didn't inherit --no-install-recommends. The new form is correct, readable, and lint-clean.
• Move COPY . . after all pip install steps — the COPY . . instruction was previously placed before pip install django-prometheus gunicorn, which busted the Docker layer cache whenever any source file changed (even for packages that don't depend on app code). Moving COPY later means pip install layers are cached independently of source changes, speeding up iterative builds.
• Remove standalone pip install vastpy — vastpy was installed outside requirements.txt in its own layer. It has been moved into requirements.txt so dependency management is centralized and the install is cacheable alongside other packages.

requirements.txt changes

• django-q2 1.5.1 → 1.7.6 — patch/minor upgrade to the async task queue library.
• ipython 7.16.3 → 8.39.0 — major upgrade; IPython 8.x requires Python ≥ 3.8 and drops several deprecated APIs. Brings in security fixes and modern shell improvements.
• vastpy added — moved from a bare pip install in the Dockerfile into requirements.txt.
• python-dateutil removed — no longer listed as a direct dependency; it is pulled in transitively by other packages (pandas, etc.).

---

Test Plan

1. Local Docker build

# Clean build (no cache) to confirm the new apt-get layer is correct
docker build --no-cache -t coldfront:test .

# Dev-mode build to confirm the conditional pip install still fires
docker build --no-cache --build-arg BUILD_ENV=dev -t coldfront:dev-test .

Verify:

• Build completes without errors
• docker run --rm coldfront:test python -c "import vastpy; import IPython; print(IPython.__version__)" prints 8.x.x
• docker run --rm coldfront:dev-test python -c "import debug_toolbar" succeeds (dev extras present)

2. Layer cache check

After the initial build, make a trivial source-file change (e.g., touch a .py file) and rebuild:

docker build -t coldfront:test .

Confirm that the pip install layers are cache hits and only the COPY . . layer and later are re-run. This validates the cache-ordering fix.

3. django-q2 upgrade smoke test

docker compose up -d
docker compose exec coldfront python manage.py qcluster &   # start task worker
docker compose exec coldfront python manage.py shell -c "
from django_q.tasks import async_task
async_task('math.floor', 1.5)
print('async_task enqueued OK')
"

4. IPython upgrade check

docker compose exec coldfront python manage.py shell
# Verify the shell starts, tab-completion works, no deprecation errors on startup

5. Full test suite

docker compose exec coldfront python manage.py test

All tests must pass before merging.

6. Staging deploy

Before pushing to production:

1. Deploy the new image to the staging environment.
2. Run a full login + allocation request workflow through the UI.
3. Confirm background task queue (django-q2) is processing jobs.
4. Check docker compose logs coldfront for any import errors or deprecation warnings at startup.
5. Verify VAST plugin functionality (since vastpy install path changed).