feat(store): migration-overlay seam — layer extra migration roots on core - #101
Merged
Merged
Conversation
…core Let the migration runner apply additional migration roots on top of migrations/core so a downstream consumer can add its own tables without editing core's migration tree: - Store.run_migrations(migrations_dir=None, overlay_dirs=None) — applies core first, then each overlay root in order. - register_migration_overlay(path) — registers an overlay root so the no-arg run_migrations() call in the mcp_http lifespan picks it up, with no call-site edit. - Overlay tracking ids are namespaced by the root's directory name so a core and an overlay file with the same filename can't collide on the schema_migrations primary key; core ids stay bare filenames. Additive and no-op by default: with no overlays a run applies exactly migrations/core and already-migrated DBs are byte-identical. The runner stays idempotent, order-deterministic, and resume-safe (overlays are recorded in schema_migrations the same way as core migrations). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
vishalkalbi27
requested review from
ashwin-agami and
sandeep-agami
as code owners
July 10, 2026 04:47
|
All contributors have signed the CLA. Thank you! |
Collaborator
Author
|
I have read the CLA Document and I hereby sign the CLA |
Collaborator
Author
|
recheck |
There was a problem hiding this comment.
Pull request overview
This PR adds a “migration overlay” seam to the Store migration runner, allowing downstream consumers to layer additional migration roots on top of migrations/core (core first, then overlays in order) without editing the core migration tree.
Changes:
- Introduces a module-level overlay registry (
register_migration_overlay) so no-argStore.run_migrations()can pick up overlays without changing existing call sites. - Extends
Store.run_migrations()to apply core migrations first, then each overlay root, using namespaced tracking IDs for overlays to avoid core/overlay filename collisions. - Adds tests covering the no-overlay path, ordering/idempotency, collision avoidance, resume behavior, and the registered-overlay no-arg path.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
packages/agami-core/src/store.py |
Adds overlay registration and extends run_migrations to support overlay roots and namespaced migration IDs. |
tests/test_auto_migrate.py |
Adds coverage for overlay behavior, ordering, id collision avoidance, resume safety, and registered overlays. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
143
to
+147
| migrations_dir = migrations_dir or MIGRATIONS_DIR | ||
| overlays = overlay_dirs if overlay_dirs is not None else list(_MIGRATION_OVERLAYS) | ||
| # (namespace, root): core is un-namespaced (bare ids, backwards-compatible); each overlay is | ||
| # namespaced by its directory name so its ids can't collide with core's on the pk. | ||
| roots = [("", migrations_dir)] + [(root.name, root) for root in overlays] |
Comment on lines
+165
to
+171
| self._run_script(path.read_text()) | ||
| self.execute( | ||
| "INSERT INTO schema_migrations (id, applied_at) VALUES (?, ?)", | ||
| (mid, _now_iso()), | ||
| ) | ||
| self.commit() | ||
| ran.append(mid) |
Overlay tracking ids are namespaced by the root's directory name. An empty name falls back to a bare id that collides with core, and two overlay roots sharing a name map distinct files to the same id — either would surface mid-apply as a schema_migrations pk violation or a skipped migration. run_migrations now validates overlay namespaces up front (before any lock or DDL) and raises a clear ValueError on an empty or duplicated overlay directory name — fail-fast, no partial apply. Docstrings updated from "should have distinct names" to reflect the enforcement. No behavior change on the no-overlay/default path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment on lines
145
to
+163
| migrations_dir = migrations_dir or MIGRATIONS_DIR | ||
| overlays = overlay_dirs if overlay_dirs is not None else list(_MIGRATION_OVERLAYS) | ||
| # Overlay tracking ids are namespaced by the root's directory name. Fail fast (before any lock | ||
| # or DDL) if a name is empty — an empty name falls back to a BARE id that collides with core — | ||
| # or duplicated across overlays — two roots would then map distinct files to the same id. Left | ||
| # unchecked, either surfaces mid-apply as a schema_migrations pk violation or a skipped migration. | ||
| namespaces = [root.name for root in overlays] | ||
| if "" in namespaces: | ||
| raise ValueError( | ||
| "overlay migration root has an empty directory name; give each overlay a distinct name" | ||
| ) | ||
| dupes = sorted({n for n in namespaces if namespaces.count(n) > 1}) | ||
| if dupes: | ||
| raise ValueError( | ||
| f"overlay migration roots share a directory name: {dupes}; names must be distinct" | ||
| ) | ||
| # (namespace, root): core is un-namespaced (bare ids, backwards-compatible); each overlay is | ||
| # namespaced by its directory name so its ids can't collide with core's on the pk. | ||
| roots = [("", migrations_dir)] + [(root.name, root) for root in overlays] |
Comment on lines
+53
to
+54
| if path not in _MIGRATION_OVERLAYS: | ||
| _MIGRATION_OVERLAYS.append(path) |
…ths on registration - run_migrations raises if a registered overlay root doesn't exist or isn't a directory — a bad path would otherwise glob to empty and silently apply nothing (unmigrated schema, no signal). - register_migration_overlay normalizes the path (resolve) before the dedup check, so different spellings of one directory (relative/absolute/symlinked) collapse to a single overlay instead of slipping through as duplicates the namespace guard would then reject. - tests: non-existent overlay raises; different spellings dedupe to one. No behavior change on the no-overlay/default path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Lets the migration runner apply additional migration roots on top of
migrations/core, in deterministic order, so a downstream consumer can add its own tables without editing core's migration tree. Additive and no-op by default.Changes
Store.run_migrations(migrations_dir=None, overlay_dirs=None)— applies core first, then each overlay root in order.register_migration_overlay(path)— registers an overlay root so the no-argrun_migrations()call in themcp_httplifespan picks it up, with no call-site edit.schema_migrationsprimary key. Core ids stay bare filenames.Backwards compatibility
migrations/core— already-migrated DBs are byte-identical (core ids unchanged).mcp_httplifespan,model_deploy) are unchanged no-arg calls.Correctness
Idempotent (a second run is a no-op), order-deterministic (core → overlay-1 → overlay-2 …), collision-safe via namespacing, and resume-safe (overlays recorded in
schema_migrationslike core migrations).Tests
tests/test_auto_migrate.pyadds: bare-core-ids on the no-overlay path, core-then-overlay + idempotency, deterministic ordering, same-filename collision avoidance, resume after a later-added overlay, and the registered-overlay no-arg path.ruffclean; the migration suite passes in isolation.