Skip to content

Conversation

@rikukissa
Copy link
Member

@rikukissa rikukissa commented Oct 21, 2025

We encounted issues with the restore script when testing backup + restore with Ashikul. This should now fix all of them

@github-actions
Copy link
Contributor

Oops! Looks like you forgot to update the changelog. When updating CHANGELOG.md, please consider the following:

  • Changelog is read by country implementors who might not always be familiar with all technical details of OpenCRVS. Keep language high-level, user friendly and avoid technical references to internals.
  • Answer "What's new?", "Why was the change made?" and "Why should I care?" for each change.
  • If it's a breaking change, include a migration guide answering "What do I need to do to upgrade?".

@rikukissa rikukissa force-pushed the backup-restore-fixes branch from bdfe75c to d72b866 Compare October 21, 2025 08:32
--network=$NETWORK \
postgres:17.6 \
bash -c "psql -h postgres -U $POSTGRES_USER -c 'DROP DATABASE IF EXISTS events;'"
bash -c "psql -h postgres -U $POSTGRES_USER -c 'DROP DATABASE IF EXISTS events WITH (FORCE);'"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@naftis I had to add a force drop here as there might be clients connected to the DB when this restore procedure is ran. Is this ok?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm feeling confident that line was there. It was added by @cibelius about month ago.

I was reviewer on that PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

might be a merge issue once again

Copy link
Contributor

@cibelius cibelius Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm feeling confident that line was there. It was added by @cibelius about month ago.

I added WITH (FORCE) to clear-all-data.sh but not this restore.sh:
https://github.com/opencrvs/opencrvs-countryconfig/blob/develop/infrastructure/clear-all-data.sh#L147-L148

Copy link
Member

@naftis naftis Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are the services restarted on restore? Not sure to be honest if we have any long running clients - but they'll get dropped then. At least restart solves it

To be honest it's a database drop so if someone is listening they'll get some funky errors any way

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I think we actually do have in our documentation https://documentation.opencrvs.org/setup/3.-installation/3.3-set-up-a-server-hosted-environment/4.3.7-backup-and-restore/4.3.7.1-restoring-a-backup

OpenCRVS requires to be re-deployed to function properly once a backup has been restored.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was looking into wrong file, sorry for misleading you

--network=$NETWORK \
postgres:17.6 \
bash -c "createdb -h postgres -U $POSTGRES_USER events && pg_restore -h postgres -U $POSTGRES_USER -d events /backups/events-${LABEL}.dump"
bash -c "createdb -h postgres -U $POSTGRES_USER events && \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to also create the database back here, with the schema and correct grants @naftis

psql -h postgres -U $POSTGRES_USER -d events -c 'CREATE SCHEMA app AUTHORIZATION events_migrator; GRANT USAGE ON SCHEMA app TO events_app;' && \
pg_restore -h postgres -U $POSTGRES_USER -d events --schema=app /backups/events-${LABEL}.dump"
echo "Update credentials in Postgres on restore"
docker service update --force opencrvs_postgres-on-update
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This updates PostgreSQL credentials after restore

--network=$NETWORK \
postgres:17.6 \
bash -c "psql -h postgres -U $POSTGRES_USER -c 'DROP DATABASE IF EXISTS events;'"
bash -c "psql -h postgres -U $POSTGRES_USER -c 'DROP DATABASE IF EXISTS events WITH (FORCE);'"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm feeling confident that line was there. It was added by @cibelius about month ago.

I was reviewer on that PR.

@rikukissa rikukissa merged commit 1ca8a47 into develop Oct 21, 2025
2 checks passed
@Nil20 Nil20 deleted the backup-restore-fixes branch October 21, 2025 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants