NOTE: This document is deprecated. It will be updated with the new process in due time.
Postgres instances are hosted within GPaaS and exist as backing services. GPaaS has a number of options available to help with backing up and restoring the database.
This process has been used to take real production data and add it to non-production environments. This is useful to test against realistic data sets.
For this example we will add production data to the staging environment.
You will need to be a user of the services GPaaS account. You're OK if you can sign into https://cloud.service.gov.uk, see the RODA service and the space you're intending to work on, eg prod
and staging
. Any member of the development team should be able to invite you.
- Sign into the platform from your machine
cf login
- Install the local Conduit plugin for Cloudfoundry. This step warns of untrusted packages. We trust that the author is the Government Digital Service and confirm
cf install-plugin conduit
- Create a new backup of production
cf target -s prod cf services # to get the POSTGRES_SERVICE_NAME of postgres cf conduit POSTGRES_SERVICE_NAME -- pg_dump --file PROD_DATA_FILE_NAME.sql
- Create a backup of the database we are about to overwrite and keep this as a local file
cf target -s staging cf services # to get the POSTGRES_SERVICE_NAME cf conduit POSTGRES_SERVICE_NAME -- pg_dump --file STAGING_DATA_FILE_NAME.sql
- Databases are linked to the app and worker service so we must find and unlink them
cf unbind-service beis-roda-staging POSTGRES_SERVICE_NAME cf unbind-service beis-roda-staging-worker POSTGRES_SERVICE_NAME
- Destroy the existing database (this may take 10 minutes)
cf delete-service POSTGRES_SERVICE_NAME
- Use Terraform to recreate it (this may take 10-20 minutes)
terraform apply --var-file=staging.tfvars
- Find the new database name
cf conduit POSTGRES_SERVICE_NAME -- psql \l (It will start with "rdsbroker_")
- Copy production data into the fresh database
cf conduit POSTGRES_SERVICE_NAME -- psql -d NEW_DATABASE_NAME < ../PROD_DATA_FILE_NAME.sql
- Seed the environment with organisations and users using the appropriate seed files
load File.join(Rails.root, "db", "seeds", "organisations.rb") load File.join(Rails.root, "db", "seeds", "pentest_users.rb")
You'll need to have credentials for the BEIS GPaaS account. (GOV.UK Platform-as-a-Service)
A download link for the Cloud Foundry command line tool appears once you are logged into GPaaS at https://www.cloud.service.gov.uk/. After downloading it, open it in Finder rather than double-clicking it directly, because there may be a warning that it is infrequently downloaded that cannot be bypassed when opening it from the browser (a second, grey, button will appear to allow you to install it anyway.)
CF Conduit is a Cloud Foundry plugin written by the GPaaS team to support the GPaaS service.
To install the plugin:
cf install-plugin conduit
This step warns of untrusted packages, we trust that the author is the Government Digital Service.
To overwrite your local development environment with the contents of a live environment, and seed the database with fake local users, you can run this command:
script/db-restore ENVIRONMENT
(Where ENVIRONMENT
is one of pentest
, prod
or staging
- default is
staging
, but prod
may be more appropriate.