Skip to content

Migrate data from mariadb to postgres#197

Merged
tilen1976 merged 47 commits intonais-migrationfrom
feat/migrate-database-to-postgres-with-flyway
May 7, 2025
Merged

Migrate data from mariadb to postgres#197
tilen1976 merged 47 commits intonais-migrationfrom
feat/migrate-database-to-postgres-with-flyway

Conversation

@tilen1976
Copy link
Contributor

@tilen1976 tilen1976 commented Apr 30, 2025

  • Properties file for postgres-local
    • Some may be helpful with Nais postgresql, but maybe not all
    • Flyway is not used
  • Docker compose for mariadb, postgresql and klass-api
  • Use profile spring.profiles.active=api, postgres-local, skip-indexing, embedded-solr in application.properties
  • Script init-migration.sh:
    • containers are created
    • data is loaded from dump file to mariadb database
    • data is migrated from mariadb to postgresql with pgloader
    • necessary alterations on datatype 'timestamp' for removing microseconds from columns
    • Dump postgresql to file
  • Script ìnit-klass-api.sh`:
    • Depends on init-migration.sh
    • Run if you want to test postgresql with klass-api
    • Klass api container is created in init-migration.sh, but application will stop because of mariadb and postgresql services not completed their jobs(no db) (depends_on only waits for containers to be ready)

Changes in code required by postgresql:

  • Remove annotation Lob (type does not exist in Postgres)
  • Set user table name in Table annotation. Table name in Postgresql must be lower case
  • Change annotation Transactional readOnly value on class ClassificationServiceImpl.java from 'true' to 'false' (default value). Instead set readOnly = true on read methods. Postgresql is more strict than mariadb and overriding value on methods in class did not work.

ref: https://statistics-norway.atlassian.net/browse/DPMETA-856

ref: https://statistics-norway.atlassian.net/wiki/x/NgCmFQE

@tilen1976 tilen1976 merged commit 700130c into nais-migration May 7, 2025
8 checks passed
@tilen1976 tilen1976 deleted the feat/migrate-database-to-postgres-with-flyway branch May 7, 2025 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants