Skip to content

Improvement analytics database lifecycle - #242

Open
jemacchi wants to merge 2 commits into
georchestra:mainfrom
jemacchi:improvement-analytics-database-lifecycle
Open

jemacchi wants to merge 2 commits into
georchestra:mainfrom
jemacchi:improvement-analytics-database-lifecycle

Conversation

@jemacchi

Copy link
Copy Markdown
Contributor

Summary

This PR integrates the Analytics database lifecycle into the shared geOrchestra Helm chart.

It complements georchestra/analytics#34 and follows the design described in GIP #25.

Changes

  • Add a revision-scoped Kubernetes Job for Alembic migrations.
  • Support safe adoption of existing pre-Alembic databases at revision 0001.
  • Add database revision gates for Vector and Analytics CLI workloads.
  • Add a recovery CronJob for the durable incremental OGC aggregation queue.
  • Synchronize configurable TimescaleDB chunk and retention policies after migration.
  • Avoid mounting legacy SQL initialization scripts when Alembic manages the schema.
  • Update the built-in database to TimescaleDB 2.30.2-pg18.
  • Update the Analytics CLI image to 2.2.0.
  • Document fresh installations, existing-database adoption and incremental aggregation.

Safe defaults

Database lifecycle operations are deliberately disabled by default:

cli:
  migrations:
    enabled: false
  aggregation:
    enabled: false

Operators must review the target database, take any required backup and explicitly enable these features during deployment:

cli:
  migrations:
    enabled: true
    targetRevision: head
  aggregation:
    enabled: true

For the first deployment over an existing pre-Alembic installation:

cli:
  migrations:
    enabled: true
    existingDatabase:
      enabled: true
      baselineRevision: "0001"

The baseline command validates the expected schema before stamping it and does not move an already-versioned database backwards.

Validation

The chart was validated with:

  • helm lint;
  • default rendering with lifecycle operations disabled;
  • opt-in rendering with migrations, revision gates and aggregation enabled;
  • legacy rendering with the original SQL initialization scripts;
  • a real installation on a disposable k3d cluster;
  • a fresh Alembic migration to revision 0003;
  • Vector startup after the database revision gate;
  • a manual incremental aggregation run;
  • a second, idempotent Helm upgrade.

The cross-repository lifecycle validation uses the Analytics application checkout to build the CLI image and this repository to deploy the shared chart. The complete workflow passed successfully.

Add revision-scoped migrations, workload revision gates, durable OGC aggregation recovery, configurable TimescaleDB policies, and upgrade documentation.
Disable Alembic migrations and incremental OGC aggregation by default. Operators must review the target database state, take any required backup, and enable both features explicitly during deployment.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant