Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,16 @@
= Migrating from internal {Project} databases to external databases

[role="_abstract"]
When you install {ProjectName}, the *{foreman-installer}* command installs PostgreSQL databases on the same server as {Project}.
If you are using the default internal databases but want to start using external databases to help with the server load, you can migrate your internal databases to external databases.
When you install {ProjectName}, PostgreSQL runs on the same server as {Project} by default.
If you use internal databases but need to reduce server load, you can migrate them to external databases.

ifdef::containerized[]
By default, `{foremanctl}` deploys internal PostgreSQL as a Podman container on {ProjectServer}.
endif::containerized[]

ifndef::containerized[]
By default, the *{foreman-installer}* command installs PostgreSQL databases on the same server as {Project}.
endif::containerized[]

ifdef::satellite[]
[IMPORTANT]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,37 @@
= Determining whether your {ProjectServer} uses internal or external databases

[role="_abstract"]
You can confirm whether your {ProjectServer} uses internal or external databases by checking the status of the PostgreSQL database service.
You can confirm whether your {ProjectServer} uses internal or external databases by checking PostgreSQL deployment on the server.

.Procedure
ifdef::containerized[]
. On {ProjectServer}, inspect the persisted database mode:
+
[options="nowrap" subs="+quotes,attributes"]
----
# grep database_mode /var/lib/foremanctl/parameters.yaml
----
+
If the output shows `database_mode: internal`, {ProjectServer} uses the default internal PostgreSQL deployment.
If the output shows `database_mode: external`, {ProjectServer} is configured to use an external PostgreSQL host.
. Optional: Confirm the status of the local PostgreSQL container service:
+
[options="nowrap" subs="+quotes,attributes"]
----
# systemctl status postgresql.service
----
+
When {ProjectServer} uses internal databases, the `postgresql.service` unit is active.
When {ProjectServer} uses external databases, the local `postgresql.service` unit is inactive or not enabled.
+
This check reflects steady-state configuration.
During a migration to external databases, `database_mode` in `/var/lib/foremanctl/parameters.yaml` may already show `external` while the local `postgresql.service` unit is still active until you remove the internal PostgreSQL deployment.
endif::containerized[]
ifndef::containerized[]
* On {ProjectServer}, query the status of your databases:
+
[options="nowrap" subs="+quotes,attributes"]
----
# {foreman-maintain} service status --only postgresql
----
endif::containerized[]
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

.Prerequisites
* The prepared host has base operating system repositories enabled.
* The prepared host has sufficient disk space available for the `{postgresql-lib-dir}` directory.
* The prepared host has enough disk space available for the `{postgresql-lib-dir}` directory.
The expected installation size is 100 MB and the expected runtime size is 20 GB.

.Procedure
Expand Down Expand Up @@ -51,7 +51,7 @@
# vi {postgresql-conf-dir}/postgresql.conf
----
+
Note that the default configuration of external PostgreSQL needs to be adjusted to work with {Project}.
Note that you must adjust the default configuration of external PostgreSQL to work with {Project}.
The base recommended external database configuration adjustments are as follows:

* checkpoint_completion_target: 0.9
Expand All @@ -65,7 +65,7 @@
----
listen_addresses = '*'
----
. Add the following line to the end of the file to use SCRAM for authentication:
. Add the following line to the end of the file to use Salted Challenge Response Authentication Mechanism (SCRAM) for authentication:

Check failure on line 68 in guides/common/modules/proc_installing-postgresql-as-an-external-database.adoc

View workflow job for this annotation

GitHub Actions / lint style

[vale] reported by reviewdog 🐶 [foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps. Raw Output: {"message": "[foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps.", "location": {"path": "guides/common/modules/proc_installing-postgresql-as-an-external-database.adoc", "range": {"start": {"line": 68, "column": 1}}}, "severity": "ERROR"}
+
[options="nowrap" subs="verbatim,quotes"]
----
Expand Down Expand Up @@ -127,7 +127,16 @@
CREATE DATABASE pulpcore OWNER pulp;
endif::[]
----
+
ifdef::containerized[]
[NOTE]
====
When you migrate from an internal database, `{foremanctl} backup` creates a `pulp.dump` file from the internal `pulp` database.
Use the `pulpcore` database name on the external PostgreSQL host, and specify `--pulp-database-name=pulpcore` when you run `{foremanctl} deploy` during migration.
A fresh containerized installation uses the `pulp` database name by default.
====
endif::containerized[]
. Exit the `postgres` user:

Check failure on line 139 in guides/common/modules/proc_installing-postgresql-as-an-external-database.adoc

View workflow job for this annotation

GitHub Actions / lint style

[vale] reported by reviewdog 🐶 [foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps. Raw Output: {"message": "[foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps.", "location": {"path": "guides/common/modules/proc_installing-postgresql-as-an-external-database.adoc", "range": {"start": {"line": 139, "column": 1}}}, "severity": "ERROR"}
+
[options="nowrap" subs="verbatim,quotes"]
----
Expand Down
126 changes: 125 additions & 1 deletion guides/common/modules/proc_migrating-to-external-databases.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,67 @@
= Migrating to external databases

[role="_abstract"]
Back up and transfer existing data, then use the `{foreman-installer}` command to configure {Project} to connect to an external PostgreSQL database server.
Migrate internal PostgreSQL databases from {ProjectServer} to a dedicated external host when you need to reduce server load or manage the database independently.

ifdef::containerized[]
[NOTE]
====
{insights-iop} requires internal database mode.
You cannot migrate to an external database while {insights-iop} is enabled on {ProjectServer}.
====
endif::containerized[]

.Prerequisites
* You have installed and configured a PostgreSQL server on an external server.
* You have confirmed that {ProjectServer} currently uses internal databases.
For more information, see xref:determining-whether-your-{project-context}-server-uses-internal-or-external-databases[Determining whether your {ProjectServer} uses internal or external databases].
* You have scheduled a maintenance window.
ifdef::containerized[]
The `{foremanctl} backup` command stops all {Project} services during the backup process.
endif::containerized[]

.Procedure
ifdef::containerized[]
. On {ProjectServer}, create a directory for the backup:

Check failure on line 28 in guides/common/modules/proc_migrating-to-external-databases.adoc

View workflow job for this annotation

GitHub Actions / lint style

[vale] reported by reviewdog 🐶 [foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps. Raw Output: {"message": "[foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps.", "location": {"path": "guides/common/modules/proc_migrating-to-external-databases.adoc", "range": {"start": {"line": 28, "column": 1}}}, "severity": "ERROR"}
+
[options="nowrap", subs="+quotes,attributes"]
----
# mkdir /var/_My_Migration_Backup_Directory_
----
. Back up the internal databases:

Check failure on line 34 in guides/common/modules/proc_migrating-to-external-databases.adoc

View workflow job for this annotation

GitHub Actions / lint style

[vale] reported by reviewdog 🐶 [foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps. Raw Output: {"message": "[foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps.", "location": {"path": "guides/common/modules/proc_migrating-to-external-databases.adoc", "range": {"start": {"line": 34, "column": 1}}}, "severity": "ERROR"}
+
[options="nowrap", subs="+quotes,attributes"]
----
ifdef::foreman-el,foreman-deb[]
# {foremanctl} backup /var/_My_Migration_Backup_Directory_
endif::[]
ifndef::foreman-el,foreman-deb[]
# {foremanctl} backup /var/_My_Migration_Backup_Directory_ \
--skip-pulp-content
endif::[]
----
+
The backup stops all {Project} services, creates database dumps, and then restarts the services.
The command creates a timestamped subdirectory, for example `/var/_My_Migration_Backup_Directory_/foreman-backup-_TIMESTAMP_/`, that contains `foreman.dump`, `candlepin.dump`, and `pulp.dump` files.
+
[NOTE]
====
The backup fails if Foreman or Pulp tasks are running.
Wait for active tasks to finish before you run the backup, or add the `--wait-for-tasks` flag to wait for running tasks to complete.
====
. Verify the backup:

Check failure on line 55 in guides/common/modules/proc_migrating-to-external-databases.adoc

View workflow job for this annotation

GitHub Actions / lint style

[vale] reported by reviewdog 🐶 [foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps. Raw Output: {"message": "[foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps.", "location": {"path": "guides/common/modules/proc_migrating-to-external-databases.adoc", "range": {"start": {"line": 55, "column": 1}}}, "severity": "ERROR"}
+
[options="nowrap", subs="+quotes,attributes"]
----
# ls -lh /var/_My_Migration_Backup_Directory_/foreman-backup-\*/
# cat /var/_My_Migration_Backup_Directory_/foreman-backup-*/metadata.yml
# ls -lh /var/_My_Migration_Backup_Directory_/foreman-backup-\*/*.dump
----
+
Confirm that the timestamped backup directory contains `foreman.dump`, `candlepin.dump`, `pulp.dump`, and `metadata.yml` files.
endif::containerized[]
ifndef::containerized[]
. On {ProjectServer}, stop all {Project} services except for PostgreSQL:

Check failure on line 67 in guides/common/modules/proc_migrating-to-external-databases.adoc

View workflow job for this annotation

GitHub Actions / lint style

[vale] reported by reviewdog 🐶 [foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps. Raw Output: {"message": "[foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps.", "location": {"path": "guides/common/modules/proc_migrating-to-external-databases.adoc", "range": {"start": {"line": 67, "column": 1}}}, "severity": "ERROR"}
+
[options="nowrap", subs="+quotes,attributes"]
----
Expand All @@ -33,16 +87,85 @@
endif::[]
/var/_My_Migration_Backup_Directory_
----
endif::containerized[]
. Transfer the data to the new external databases:

Check failure on line 91 in guides/common/modules/proc_migrating-to-external-databases.adoc

View workflow job for this annotation

GitHub Actions / lint style

[vale] reported by reviewdog 🐶 [foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps. Raw Output: {"message": "[foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps.", "location": {"path": "guides/common/modules/proc_migrating-to-external-databases.adoc", "range": {"start": {"line": 91, "column": 1}}}, "severity": "ERROR"}
+
[options="nowrap", subs="+quotes,attributes"]
----
ifdef::containerized[]
PGPASSWORD='_Foreman_Password_' pg_restore --host=_postgres.example.com_ --username=foreman --dbname=foreman < /var/_My_Migration_Backup_Directory_/foreman-backup-_TIMESTAMP_/foreman.dump
ifndef::foreman-el,foreman-deb[]
PGPASSWORD='_Candlepin_Password_' pg_restore --host=_postgres.example.com_ --username=candlepin --dbname=candlepin < /var/_My_Migration_Backup_Directory_/foreman-backup-_TIMESTAMP_/candlepin.dump
PGPASSWORD='_Pulpcore_Password_' pg_restore --host=_postgres.example.com_ --username=pulp --dbname=pulpcore < /var/_My_Migration_Backup_Directory_/foreman-backup-_TIMESTAMP_/pulp.dump
endif::[]
endif::containerized[]
ifndef::containerized[]
PGPASSWORD='_Foreman_Password_' pg_restore --host=_postgres.example.com_ --username=foreman --dbname=foreman < /var/_My_Migration_Backup_Directory_/foreman.dump
ifndef::foreman-el,foreman-deb[]
PGPASSWORD='_Candlepin_Password_' pg_restore --host=_postgres.example.com_ --username=candlepin --dbname=candlepin < /var/_My_Migration_Backup_Directory_/candlepin.dump
PGPASSWORD='_Pulpcore_Password_' pg_restore --host=_postgres.example.com_ --username=pulp --dbname=pulpcore < /var/_My_Migration_Backup_Directory_/pulpcore.dump
endif::[]
endif::containerized[]
----
ifdef::containerized[]
+
Replace `foreman-backup-_TIMESTAMP_` with the timestamp directory that the backup created.
endif::containerized[]

ifdef::containerized[]
. Use the `{foremanctl} deploy` command to update {Project} to point to the new external databases:

Check failure on line 116 in guides/common/modules/proc_migrating-to-external-databases.adoc

View workflow job for this annotation

GitHub Actions / lint style

[vale] reported by reviewdog 🐶 [foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps. Raw Output: {"message": "[foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps.", "location": {"path": "guides/common/modules/proc_migrating-to-external-databases.adoc", "range": {"start": {"line": 116, "column": 1}}}, "severity": "ERROR"}
+
[options="nowrap", subs="+quotes,attributes"]
----
# {foremanctl} deploy \
--database-mode external \
--database-host _postgres.example.com_ \
--database-port 5432 \
--foreman-database-name foreman \
--foreman-database-user foreman \
ifdef::foreman-el,foreman-deb[]
--foreman-database-password _Foreman_Password_
endif::[]
ifndef::foreman-el,foreman-deb[]
Comment thread
Lennonka marked this conversation as resolved.
--foreman-database-password _Foreman_Password_ \
--candlepin-database-name candlepin \
--candlepin-database-user candlepin \
--candlepin-database-password _Candlepin_Password_ \
--pulp-database-name pulpcore \
--pulp-database-user pulp \
--pulp-database-password _Pulpcore_Password_
endif::[]
----
+
For a complete list of external database parameters, including optional SSL connection options, see {InstallingServerDocURL}deploy-{project-context}-server-configuration-with-an-external-database[Deploy {ProjectServer} configuration with an external database] in _{InstallingServerDocTitle}_.
. Verify that {ProjectServer} uses the external databases and that {Project} is operational:

Check failure on line 141 in guides/common/modules/proc_migrating-to-external-databases.adoc

View workflow job for this annotation

GitHub Actions / lint style

[vale] reported by reviewdog 🐶 [foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps. Raw Output: {"message": "[foreman-documentation.ProcedureStepLimit] Procedure contains more than 10 steps. Split it into multiple procedures or leverage substeps to group related steps. If the procedure contains conditionals, review the preview to ensure all built targets comply with the limit of 10 steps.", "location": {"path": "guides/common/modules/proc_migrating-to-external-databases.adoc", "range": {"start": {"line": 141, "column": 1}}}, "severity": "ERROR"}
+
[options="nowrap", subs="+quotes,attributes"]
----
# grep database_mode /var/lib/foremanctl/parameters.yaml
# {foremanctl} health
$ hammer ping
----
+
Confirm that `database_mode` is set to `external`, that `{foremanctl} health` reports no failures, and that `hammer ping` returns a successful response before you remove the internal PostgreSQL deployment.
. Remove the obsolete internal PostgreSQL deployment on {ProjectServer}:
+
The internal PostgreSQL database runs as a Podman container managed by the `postgresql.service` systemd unit.
+
[options="nowrap", subs="+quotes,attributes"]
----
# systemctl stop postgresql.service
# systemctl disable postgresql.service
# rm -rf /var/lib/pgsql/data
----
+
[NOTE]
====
When you switch to external databases, `{foremanctl} deploy` does not remove Podman quadlet unit files under `/etc/containers/systemd/`, such as `postgresql.container` and `postgresql.image`, or Podman secrets, such as `postgresql-admin-password`.
Stopping the service and removing `/var/lib/pgsql/data` disables the internal database and frees the data volume.
====
endif::containerized[]
ifndef::containerized[]
. Use the `{foreman-installer}` command to update {Project} to point to the new
ifndef::foreman-deb,foreman-el[]
databases:
Expand Down Expand Up @@ -88,3 +211,4 @@
+
If you run {ProjectServer} on Debian 12, you have to adjust the paths of PostgreSQL config and data directories which depend on the PostgreSQL version.
endif::[]
endif::containerized[]
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,21 @@

[role="_abstract"]
ifdef::katello,orcharhino,satellite[]
Foreman, Katello, and Candlepin use the PostgreSQL database.
Foreman, Katello, and Candlepin use PostgreSQL.
endif::[]
ifndef::katello,orcharhino,satellite[]
Foreman uses the PostgreSQL database.
Foreman uses PostgreSQL.
endif::[]
If you want to use PostgreSQL as an external database, the following information can help you decide if this option is right for your {Project} configuration.
Review the advantages and disadvantages of external PostgreSQL to decide if this option fits your {Project} configuration.

ifdef::foreman-deb[]
{Project} requires PostgreSQL version 13 or later.
{Team} recommends that you use the PostgreSQL version that is part of the default operating system repositories.
endif::[]
ifdef::containerized[]
{Project} supports PostgreSQL version 16, which is available in {EL} 10 repositories.
By default, internal PostgreSQL runs in a Podman container on {ProjectServer}.
An external database runs on a dedicated host that you provision and maintain separately.
endif::[]
ifndef::foreman-deb,containerized[]
{Project} supports PostgreSQL version 13.
Expand Down
4 changes: 2 additions & 2 deletions guides/doc-Administering_Project/master.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,10 @@ ifndef::containerized[]
include::common/assembly_tuning-with-predefined-profiles.adoc[leveloffset=+1]
endif::[]

ifndef::containerized[]

include::common/assembly_migrating-from-internal-databases-to-external-databases.adoc[leveloffset=+1]

ifndef::containerized[]

include::common/assembly_preparing-for-disaster-recovery-and-recovering-from-data-loss.adoc[leveloffset=+1]

:parent-context: {context}
Expand Down
Loading