Skip to content

MET-MIG-V1.9-RECON-01 - Reconcile unreleased Metrics migration directory suffixes to v1.9.0 #868

Description

@ja573

MET-MIG-V1.9-RECON-01 - Reconcile unreleased Metrics migration identities to v1.9.0

Status: DRAFT - FRESH INDEPENDENT SPECIFICATION REVIEW REQUIRED / IMPLEMENTATION DEPENDENCY HOLD

Programme: Thoth Metrics - canonical ingestion, Sphinx orchestration and client cutover
Stage: release-number / migration-identity reconciliation before final Metrics integration
Parent programme issue: #766
Predecessor state: MET-WP1-01 / #836, MET-WP1-02 / #841 and MET-WP1-03 / #863 are merged to feature/metrics
Release coordination: #864 / PR #866 published Thoth v1.8.0; #867 (REL-1.8.0-DEV-RECON-01) is the separate release-forward task into develop
Task ID: MET-MIG-V1.9-RECON-01
Repository: thoth-pub/thoth
Workflow: PROGRAMME_INTEGRATION
Programme integration branch: feature/metrics
Exact specification-inspection baseline: feature/metrics @ f06230ba7bff099c0c0fb34b4747057b4a9b7a30
Observed develop at task setup: 665583371e0bbb64bf2d71836b4ce1a73e57d3a2
Published v1.8.0 / master commit: 9ae1e56714096507d2d210b5b315361d626626c9
Expected implementation base: NOT YET BOUND - DEPENDENCY HOLD
PR target after separate implementation authorization: feature/metrics
Proposed task branch after separate authorization: feature/metrics--v1.9-migration-reconcile
Risk: HIGH
Owner: CTO
Approved by: NOT YET APPROVED
Runtime effect at this gate: NONE
Production migration/deployment/activation authorization: NONE

Issue creation records a proposed bounded migration-identity reconciliation specification only. It does not authorize migration/file renames, source/document edits, branch creation, disposable-database execution, commit, push, PR creation/update, automatic staging-image publication, manual CI, provider/runtime access, staging/production migration execution, merge, release, deployment, production activation, task-branch deletion, release-forward reconciliation, feature/metrics refresh, or implementation of another Metrics slice.

1. Objective

Reconcile the release identity of the unreleased Thoth Metrics migration chain after Thoth v1.8.0 was published for the ORCID batch API hotfix.

At the exact inspection baseline, feature/metrics contains three Metrics migrations created while the workspace release baseline was 1.7.0, so their directory suffixes are v1.8.0:

thoth-api/migrations/20260826_v1.8.0/
thoth-api/migrations/20260827_v1.8.0/
thoth-api/migrations/20260828_v1.8.0/

The published Thoth release now owns version v1.8.0. Repository make migration convention increments the current minor version, so after the release-forward reconciliation brings workspace version 1.8.0 to the development line, the next migration suffix is v1.9.0.

This task therefore renames the three unreleased Metrics migration identities to:

thoth-api/migrations/20260826_v1.9.0/
thoth-api/migrations/20260827_v1.9.0/
thoth-api/migrations/20260828_v1.9.0/

while preserving every up.sql and down.sql byte-for-byte and updating only current, semantically linked forward-looking repository references that would otherwise describe the next Metrics migration chain incorrectly.

This is release-number coordination only. It changes no Metrics schema semantics, runtime behavior, GraphQL/API contract, authorization, source mapping, data, dependency or deployment state.

2. Authority and dependency ordering

Controlling durable programme records:

  1. Thoth Metrics: canonical ingestion, Sphinx orchestration and client cutover #766 comment 5460760037 established that Thoth v1.8.0 is reserved for the hotfix release line and requires Metrics migration/release references to reconcile to v1.9.0 before final integration/release.
  2. Thoth Metrics: canonical ingestion, Sphinx orchestration and client cutover #766 comment 5461128004, after actual v1.8.0 publication and MET-WP1-03 merge, expands the inventory to the three paths above and requires a separate feature/metrics refresh from develop after REL-1.8.0-DEV-RECON-01: Reconcile v1.8.0 release into develop #867.
  3. REL-1.8.0-PREP-01: Prepare Thoth v1.8.0 release #864 / PR REL-1.8.0-PREP-01: Prepare Thoth v1.8.0 release #866 define the v1.8.0 release and explicitly exclude Metrics source mutation from release preparation.
  4. Live released source at 9ae1e56714096507d2d210b5b315361d626626c9 has workspace version 1.8.0; live Makefile migration generation increments the minor version, making v1.9.0 the next migration suffix.
  5. The three Metrics migrations are absent from published v1.8.0; they remain unreleased programme-branch history.

2.1 Required ordering before implementation authorization

This specification may be reviewed now, but implementation is intentionally on HOLD until both conditions are satisfied:

  1. REL-1.8.0-DEV-RECON-01: Reconcile v1.8.0 release into develop #867 has completed the separately controlled v1.8.0 -> develop release-forward reconciliation; and
  2. a separately authorized/reviewed Metrics integration refresh has merged that updated develop state into feature/metrics.

Only then may this task bind an exact implementation base and proceed to separate CTO implementation authorization.

Rationale: doing the suffix reconciliation before the release-forward and Metrics refresh would knowingly edit a branch that is about to ingest the release-version/changelog state that motivates the rename, increasing conflict/rework risk. The refreshed base should naturally carry workspace version 1.8.0, making the v1.9.0 migration convention locally reproducible.

If the post-refresh feature/metrics delta materially changes the migration inventory, current migration references, release semantics or write budget below, return HOLD - SPECIFICATION/WRITE-BUDGET REBINDING REQUIRED and amend/re-review this specification rather than silently expanding scope.

3. Risk classification

HIGH.

The intended content change is mechanically narrow and contains no SQL/schema semantic change, but migration directory names are durable migration identities used by Diesel's migration ledger and later deployment history. A mistaken rename, partial rename, changed SQL blob or rename after a persistent environment has applied the old identity could create duplicate/replayed migration behavior or an irreconcilable migration ledger.

HIGH-risk controls therefore apply:

  • written specification and fresh independent review before implementation;
  • dependency HOLD until the release-forward and Metrics refresh are complete;
  • exact SHA-bound implementation authorization;
  • byte-identity proof for all six SQL files;
  • full migration-chain apply/revert/reapply validation on disposable PostgreSQL;
  • fresh independent exact-head source review;
  • exact-SHA CTO merge authorization;
  • no persistent-environment migration execution or history manipulation.

4. Explicit scope

4.1 Rename exactly the current three Metrics migration directories

At the specification baseline, the intended path-only moves are:

20260826_v1.8.0 -> 20260826_v1.9.0
20260827_v1.8.0 -> 20260827_v1.9.0
20260828_v1.8.0 -> 20260828_v1.9.0

for both up.sql and down.sql within each directory.

Requirements:

  • preserve the date prefix exactly;
  • preserve all six SQL file contents byte-for-byte;
  • preserve file mode;
  • do not edit SQL comments, whitespace or formatting merely to make the rename visible;
  • old paths must be absent after the rename;
  • new paths must contain the exact original blobs;
  • no additional migration is created.

Immediately before implementation authorization, freshly inventory feature/metrics for Metrics-owned next-release migration directories still carrying _v1.8.0. If more than these three exist, HOLD - MIGRATION INVENTORY / WRITE-BUDGET AMENDMENT REQUIRED rather than silently renaming additional paths.

4.2 Update current forward-looking migration-path references only

The exact specification baseline contains current path references in:

  • CHANGELOG.md for MET-WP1-01, MET-WP1-02, MET-WP1-03;
  • thoth-api/src/model/metric_platform/tests.rs doc comment for the MET-WP1-01 migration;
  • thoth-api/src/model/metric_source/tests.rs doc comment for the MET-WP1-02 migration;
  • thoth-api/src/model/metric_import/tests.rs doc comment for the MET-WP1-03 migration.

Update those current references from v1.8.0 to v1.9.0 where still present on the freshly rebound implementation base.

The executable Diesel migration-version constants remain the date IDs (20260826, 20260827, 20260828) and must not be changed merely because the directory suffix changes.

CHANGELOG.md may add one bounded Unreleased entry for this task if required by repository changelog policy, but it must not duplicate or reinterpret the three WP1 schema descriptions. The existing WP1 entries should describe their current unreleased migration paths truthfully after the rename.

4.3 Durable implementation report

Add:

docs/engineering/ai-delivery/implementation-reports/MET-MIG-V1.9-RECON-01-implementation-report.md

The report must record exact base/head, path moves, before/after SQL blob hashes, reference changes, migration-chain validation, test results, CI/PR state, external effects, limitations and remaining gates. It is evidence only and is not self-approval.

4.4 Historical provenance is immutable

Do not rewrite historical point-in-time evidence to pretend the migrations were originally created under v1.9.0.

Specifically out of scope for path-string cleanup solely for this task:

Those records remain valid historical provenance. New task/report/merge records must explain the later rename instead of rewriting history.

5. Candidate implementation write budget

This is the maximum expected write envelope for later authorization; it is not active authorization.

Current files whose contents may change:

CHANGELOG.md
thoth-api/src/model/metric_platform/tests.rs
thoth-api/src/model/metric_source/tests.rs
thoth-api/src/model/metric_import/tests.rs

Path-only moves; SQL bytes must remain identical:

thoth-api/migrations/20260826_v1.8.0/up.sql
thoth-api/migrations/20260826_v1.8.0/down.sql
thoth-api/migrations/20260827_v1.8.0/up.sql
thoth-api/migrations/20260827_v1.8.0/down.sql
thoth-api/migrations/20260828_v1.8.0/up.sql
thoth-api/migrations/20260828_v1.8.0/down.sql

into:

thoth-api/migrations/20260826_v1.9.0/up.sql
thoth-api/migrations/20260826_v1.9.0/down.sql
thoth-api/migrations/20260827_v1.9.0/up.sql
thoth-api/migrations/20260827_v1.9.0/down.sql
thoth-api/migrations/20260828_v1.9.0/up.sql
thoth-api/migrations/20260828_v1.9.0/down.sql

New evidence file:

docs/engineering/ai-delivery/implementation-reports/MET-MIG-V1.9-RECON-01-implementation-report.md

No schema.rs, Rust model implementation, Cargo manifest/lockfile, GraphQL, workflow, provider/configuration, downstream repository or other Metrics document is in the candidate budget.

If the refreshed implementation base contains another current semantically linked reference whose correction is required for repository truthfulness or tests, return HOLD - WRITE BUDGET AMENDMENT REQUIRED with the exact path and occurrence before editing it.

6. Explicit non-goals / invariants

This task must not:

  1. modify any migration SQL content or schema semantics;
  2. create a fourth migration or collapse/reorder the three migrations;
  3. change migration date prefixes or Diesel migration-version constants;
  4. modify thoth-api/src/schema.rs or any persistence/domain model;
  5. change Cargo/package versions, dependency pins or Cargo.lock - those belong to the release-forward history;
  6. implement or modify GraphQL/API behavior;
  7. change auth/security, entitlements, machine roles or source/platform mappings;
  8. seed, backfill or mutate canonical data;
  9. rewrite historical implementation/review/authorization/merge evidence;
  10. run migrations against staging, production or another persistent shared database;
  11. edit Diesel migration ledger rows to disguise an already-applied old identity;
  12. perform REL-1.8.0-DEV-RECON-01: Reconcile v1.8.0 release into develop #867 or the post-REL-1.8.0-DEV-RECON-01: Reconcile v1.8.0 release into develop #867 feature/metrics refresh inside this task;
  13. merge feature/metrics into develop;
  14. modify thoth-app, Sphinx, dissemination or another repository;
  15. deploy, tag, release, publish or activate Metrics behavior;
  16. start another WP1/WP2+ slice automatically.

Invariants:

  • the effective database schema produced by the full migration chain is unchanged;
  • all six SQL blobs are byte-identical before and after;
  • migration ordering by date prefix is unchanged;
  • WP1 remains IN PROGRESS;
  • the published Thoth release remains v1.8.0;
  • the unreleased Metrics migration chain becomes v1.9.0-identified only after this task merges;
  • implementing agent may not approve its own work.

7. Persistent-environment safety condition

The released v1.8.0 source does not contain these Metrics migrations, and no production migration was authorized by the three WP1 child tasks. That supports path renaming before release integration.

Nevertheless, if authoritative deployment/runtime evidence shows that any persistent shared staging/production database has applied one of the old directory identities (20260826, 20260827, 20260828) from this Metrics branch outside the recorded controls, return:

BLOCKED - EXISTING PERSISTENT MIGRATION HISTORY REQUIRES A SEPARATE MIGRATION-LEDGER/COMPATIBILITY PLAN

Do not rename around or manually edit persistent Diesel migration history within this task.

Disposable CI/local databases do not create this blocker.

8. Acceptance criteria

8.1 Dependency and exact-base gate

  • REL-1.8.0-DEV-RECON-01: Reconcile v1.8.0 release into develop #867 is complete and its released v1.8.0 state is reachable from live develop.
  • the separately controlled post-REL-1.8.0-DEV-RECON-01: Reconcile v1.8.0 release into develop #867 Metrics refresh is complete and live feature/metrics contains that updated develop state.
  • a fresh exact feature/metrics implementation base SHA is recorded before authorization.
  • live workspace version/migration convention on that refreshed base is consistent with v1.9.0 as the next migration suffix; otherwise HOLD.
  • no additional Metrics _v1.8.0 migration has appeared; otherwise HOLD for inventory/spec amendment.
  • proposed branch namespace preflight passes and feature/metrics--v1.9-migration-reconcile is absent before creation.

8.2 Path/blob identity

  • exactly the three approved migration directories move from _v1.8.0 to _v1.9.0.
  • old six SQL paths are absent.
  • new six SQL paths are present.
  • each new up.sql/down.sql blob/hash is byte-identical to its old counterpart.
  • no SQL content diff exists.
  • date ordering remains 20260826, 20260827, 20260828.
  • no new migration is added.

8.3 Reference integrity

  • current CHANGELOG.md WP1 migration paths identify v1.9.0.
  • the three current test doc comments identify their renamed v1.9.0 paths.
  • executable migration-version constants remain 20260826, 20260827, 20260828.
  • no current forward-looking repository reference incorrectly claims these next-release Metrics paths are v1.8.0.
  • historical reports/issues/PR/review evidence remains unchanged.

8.4 Migration/schema equivalence

Using disposable PostgreSQL only under later explicit implementation authorization:

  • full migration chain applies cleanly from empty database;
  • targeted revert/reapply behavior for MET-WP1-01/02/03 remains correct;
  • representative populated current-schema migration validation remains clean;
  • final schema/seed/data result is equivalent to the pre-rename migration chain;
  • no production/staging/shared-persistent migration is executed.

8.5 Repository validation

At minimum:

cargo test -p thoth-api --features backend
cargo test --workspace
cargo check --workspace
cargo clippy --all --all-targets --all-features -- -D warnings
cargo fmt --all -- --check
git diff --check

Also record a path-level diff proving the SQL changes are renames only with identical contents.

8.6 API/compatibility

  • public GraphQL/export behavior is unchanged.
  • no generated client change is required.
  • no downstream repository change is required.
  • no auth/security/runtime behavior changes.

9. Automatic PR side effects

A later PR containing .rs reference updates and migration path moves is expected under current workflows to trigger normal PR build/test/lint/format, migration validation, changelog validation and staging container/image publication.

The staging image publication is an automatic external registry write. PR creation and that external side effect require a separate explicit authorization after fresh workflow/classifier verification at the exact task head. Manual CI dispatch/rerun/cancel remains separately unauthorized unless explicitly granted.

10. Rollout and rollback

Repository rollout only:

  1. after dependencies complete, bind exact refreshed feature/metrics base;
  2. separately authorize bounded implementation;
  3. rename paths/update current references/validate;
  4. obtain fresh independent exact-head source review;
  5. separately authorize and merge only into feature/metrics;
  6. retain the v1.9 reconciliation as a prerequisite for later final feature/metrics -> develop integration/release.

Before these migrations are ever applied to a persistent environment, source rollback is a separately authorized revert of this bounded reconciliation.

After any v1.9.0 identity has been applied to a persistent environment, do not revert migration directory identities in isolation; use a separately reviewed migration-history/forward-repair plan.

No production rollback is authorized here.

11. HOLD / BLOCKED / STOP

HOLD

HOLD if:

  • REL-1.8.0-DEV-RECON-01: Reconcile v1.8.0 release into develop #867 is not complete;
  • the required post-REL-1.8.0-DEV-RECON-01: Reconcile v1.8.0 release into develop #867 Metrics refresh is not complete;
  • feature/metrics moves after exact implementation authorization;
  • the refreshed branch changes the three migration contents or introduces additional _v1.8.0 Metrics migrations;
  • a required current reference lies outside the candidate write budget;
  • the expected branch already exists or namespace preflight fails;
  • workspace/migration convention after refresh does not support v1.9.0;
  • migration/workflow behavior changed materially;
  • implementation would require SQL/schema/Cargo/dependency changes.

BLOCKED

BLOCKED if:

  • authoritative evidence shows an old Metrics migration identity has been applied to a persistent shared environment and safe rename semantics are no longer established;
  • release authority disagrees on whether v1.8.0 or v1.9.0 owns the Metrics migration chain;
  • path renaming cannot preserve exact SQL/schema equivalence;
  • a cross-repository or production migration-history change is required.

STOP

STOP if work begins to:

  • change migration SQL/schema semantics;
  • change package/release versioning outside the three migration suffixes and current references;
  • modify GraphQL/API/auth/runtime behavior;
  • edit persistent migration ledger state;
  • run staging/production migrations;
  • perform release-forward/branch refresh work;
  • deploy/release/activate;
  • merge feature/metrics -> develop;
  • widen into another programme or later Metrics slice.

12. Action-authorization matrix at issue creation

Specification/review only.

Action Authorized now
repository/GitHub read inspection for specification review YES
issue review/comments under later explicit control action NO
migration/file rename or source/doc modification NO
branch/ref creation or movement NO
local disposable migration execution NO
commit NO
push NO
PR create/update/ready NO
automatic GHCR staging publication NO
manual CI dispatch/rerun/cancel NO
provider/runtime read or write NO
staging/production migration execution NO
persistent migration-ledger edit NO
merge NO
release/tag/publication NO
deployment/activation NO
branch deletion NO

13. Independent review expectations

Fresh independent specification review must inspect at minimum:

  1. this complete issue;
  2. Thoth Metrics: canonical ingestion, Sphinx orchestration and client cutover #766 comments 5460760037 and 5461128004;
  3. REL-1.8.0-PREP-01: Prepare Thoth v1.8.0 release #864 / PR REL-1.8.0-PREP-01: Prepare Thoth v1.8.0 release #866 and live published v1.8.0 identity;
  4. REL-1.8.0-DEV-RECON-01: Reconcile v1.8.0 release into develop #867 current state and the dependency ordering defined here;
  5. exact live feature/metrics, develop and released v1.8.0 refs;
  6. the three migration directories and all six SQL blobs;
  7. current CHANGELOG.md and the three test doc comments named in section 4.2;
  8. historical WP1 reports/reviews to confirm they are provenance that must not be rewritten;
  9. Diesel migration identity semantics and the safety of renaming only because these migrations are unreleased/unapplied to persistent production state;
  10. exact candidate write budget, validation, PR side effects and HOLD/BLOCKED/STOP conditions.

Decision must be one of:

APPROVED
CHANGES REQUIRED
BLOCKED
HOLD

Specification approval does not remove the implementation dependency HOLD and does not authorize implementation, PR creation, GHCR publication, merge, migration execution, release, deployment or activation.

14. Current gate

MET-MIG-V1.9-RECON-01
SPECIFICATION DRAFT
INSPECTION BASELINE: feature/metrics @ f06230ba7bff099c0c0fb34b4747057b4a9b7a30
RELEASE AUTHORITY: v1.8.0 @ 9ae1e56714096507d2d210b5b315361d626626c9
OBSERVED DEVELOP: 665583371e0bbb64bf2d71836b4ce1a73e57d3a2
RISK: HIGH
IMPLEMENTATION: HOLD - #867 + SEPARATE POST-RELEASE FEATURE/METRICS REFRESH REQUIRED
NEXT: FRESH INDEPENDENT SPECIFICATION REVIEW

After independent specification approval, CTO specification approval remains separate. Implementation authorization must wait until the dependency chain is complete and must bind the exact then-current refreshed feature/metrics SHA.

Refs #766
Refs #836
Refs #841
Refs #863
Refs #864
Refs #867

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions