Skip to content

fix(packaging): ship migrations + static assets inside the wheel - #124

Merged
vishalkalbi27 merged 3 commits into
mainfrom
fix/package-static-assets-in-wheel
Jul 17, 2026
Merged

vishalkalbi27 merged 3 commits into
mainfrom
fix/package-static-assets-in-wheel

Conversation

@vishalkalbi27

Copy link
Copy Markdown
Collaborator

Summary

Fixes two failures that existed only in a real (non-editable) wheel install — never in a checkout or the Docker deploy (which uses pip install -e), which is why CI stayed green and neither was caught.

The bugs

  1. Migrations silently never applied. store.MIGRATIONS_DIR resolved via Path(__file__).parents[3] / "migrations" / "core". In a checkout that's <repo>/migrations/core ✅ — but in a wheel, store.py lands in site-packages/, so parents[3] points at the venv root. That directory doesn't exist, glob() returns [], and the server boots on an empty schema with no error at all.
  2. Static assets missing. mcp_http._STATIC_DIR = Path(__file__).parent / "static", but static/ was never listed in pyproject.toml, so it wasn't in the wheel → StaticFiles(directory=…) raised "Directory does not exist" when building the app.

The fix

  • Move migrations/core into the package (packages/agami-core/src/migrations/core) and resolve MIGRATIONS_DIR next to store.py — so it works identically from a wheel and from a checkout.
  • Package static and migrations as package-data in pyproject.toml.
  • Guard run_migrations: a missing core migration root now raises instead of silently applying nothing — this class of bug can no longer fail quietly.
  • Dockerfile: drop COPY migrations — the package source now carries them.

Verification

Not just tests — the artifact itself was built and run:

  • Wheel + sdist contents — both carry all 11 migrations/core/*.sql (+ README) and all 8 static/** assets.
  • Clean-venv wheel install, executed from /tmp (no checkout reachable):
    GET /static/logo_h.svg -> 200 image/svg+xml      # previously CRASHED
    POST /mcp (no auth)    -> 401 + WWW-Authenticate # auth challenge intact
    migrations applied at startup: 11                # previously SILENTLY 0
    
  • Docker image rebuilt with the changed Dockerfile — both dirs resolve inside the image (/app/packages/agami-core/src/{migrations/core,static}), /app/migrations is gone, and the container boots: static 200, /mcp 401, 11 migrations applied.
  • uv run dev.py check — ruff ✓, ruff-format ✓, pytest 1498 passed / 1 skipped ✓, gitleaks ✓.

Blast radius

Only the HTTP server path uses these directories:

  • mcp_http — startup migrations + the /static mount.
  • model_deploy — startup migrations.

The local plugin skills and the stdio MCP server (mcp_harness) import neither store nor the static assets, so they have no exposure to this class of bug. The Docker/editable path was never broken and still works.

Follow-up

Related to #122. This PR fixes the packaging, but the test suite still cannot catch a regression — tests install editable, so the data dirs resolve from the source tree regardless of what pyproject.toml packages. A revert of the packaging config would keep all 1498 tests green while shipping a broken wheel. #122 tracks adding that guard (plus declaring the static.fonts / migrations.core subpackages to silence the setuptools "absent from the packages configuration" warnings).

Two failures existed only in a real (non-editable) wheel install — never in a checkout or the
Docker deploy, which uses `pip install -e`, which is why CI stayed green:

- **migrations** — `store.MIGRATIONS_DIR` resolved via `Path(__file__).parents[3] / "migrations" /
  "core"`. In a checkout that's `<repo>/migrations/core`, but in a wheel `store.py` lands in
  `site-packages/`, so `parents[3]` points at the venv root. The directory doesn't exist, `glob()`
  returns `[]`, and the server boots on an EMPTY schema with no error at all.
- **static** — `mcp_http._STATIC_DIR = Path(__file__).parent / "static"`, but `static/` was never
  listed in pyproject, so it wasn't in the wheel and `StaticFiles(directory=…)` raised
  "Directory does not exist" when building the app.

Fix:

- Move `migrations/core` into the package (`packages/agami-core/src/migrations/core`) and resolve
  `MIGRATIONS_DIR` next to `store.py`, so it works identically from a wheel and a checkout.
- Package `static` and `migrations` as package-data in pyproject.
- Guard `run_migrations`: a missing core migration root now raises instead of silently applying
  nothing, so this class of bug can no longer fail quietly.
- Dockerfile: drop `COPY migrations` — the package source now carries them.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes wheel-install-only runtime failures by packaging database migrations and HTTP static assets inside the agami-core distribution and updating runtime path resolution so it works the same from an installed wheel and from an editable checkout.

Changes:

  • Move/ship migrations/core alongside the flat modules and resolve MIGRATIONS_DIR relative to store.py to avoid “silent no-op” migrations in wheel installs.
  • Add a fail-loud guard in Store.run_migrations when the core migrations root is missing/not a directory.
  • Update packaging (pyproject.toml) and Docker image build to include the packaged static/ and migrations/ assets without a separate Docker COPY.

Reviewed changes

Copilot reviewed 3 out of 15 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
packages/agami-core/src/store.py Resolves migrations relative to the installed module location and fails loudly if the core migrations directory is missing.
packages/agami-core/src/migrations/core/README.md Adds documentation placeholder for the migrations directory.
packages/agami-core/src/migrations/core/001_serving.sql Adds core schema migration file to the packaged migrations directory.
packages/agami-core/src/migrations/core/002_runtime.sql Adds core schema migration file to the packaged migrations directory.
packages/agami-core/src/migrations/core/003_users.sql Adds core schema migration file to the packaged migrations directory.
packages/agami-core/src/migrations/core/004_oauth.sql Adds core schema migration file to the packaged migrations directory.
packages/agami-core/src/migrations/core/005_users_passwordless.sql Adds core schema migration file to the packaged migrations directory.
packages/agami-core/src/migrations/core/006_oidc_identity.sql Adds core schema migration file to the packaged migrations directory.
packages/agami-core/src/migrations/core/007_user_names.sql Adds core schema migration file to the packaged migrations directory.
packages/agami-core/src/migrations/core/008_tool_calls.sql Adds core schema migration file to the packaged migrations directory.
packages/agami-core/src/migrations/core/009_tool_calls_correlation.sql Adds core schema migration file to the packaged migrations directory.
packages/agami-core/src/migrations/core/010_oauth_refresh.sql Adds core schema migration file to the packaged migrations directory.
packages/agami-core/src/migrations/core/011_query_executions_ts.sql Adds core schema migration file to the packaged migrations directory.
packages/agami-core/pyproject.toml Declares static and migrations as packaged content and includes all files as package data.
deploy/Dockerfile Removes the separate migrations copy step since migrations/static now live under the package source.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 150 to +154
migrations_dir = migrations_dir or MIGRATIONS_DIR
# Same failure mode the overlays guard against, but for CORE — and far worse: a missing core root
# globs to nothing, so the server would boot on an EMPTY schema with no error at all. Fail loudly.
if not migrations_dir.is_dir():
raise ValueError(f"core migration root is not a directory: {migrations_dir}")
…e path

The fail-loud guard for a missing core migrations root had no test — it could be removed and
nothing would fail, letting the silent-empty-schema mode regress. Adds:

- test_missing_core_migrations_root_raises: a non-directory core root raises before any DDL
  (mirrors the existing overlay-root coverage).
- test_default_core_migrations_root_lives_inside_the_package: MIGRATIONS_DIR resolves next to
  store.py and actually ships .sql files, so a revert to an out-of-tree path (which still passes
  in a checkout but yields nothing once installed) fails here.

Both were confirmed to fail when the guard and the in-package path are reverted.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@vishalkalbi27
vishalkalbi27 merged commit fd28924 into main Jul 17, 2026
6 checks passed
@vishalkalbi27
vishalkalbi27 deleted the fix/package-static-assets-in-wheel branch July 17, 2026 10:42
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants