diff --git a/README.md b/README.md index c558b8a..a7c6fd7 100644 --- a/README.md +++ b/README.md @@ -44,10 +44,10 @@ services: - PRIMARY=http://ph1.example.com|password - REPLICAS=http://ph2.example.com|password,http://ph3.example.com|password - FULL_SYNC=true - - RUN_GRAVITY=true - - CRON=0 * * * * ``` +> **Note:** This example only includes the **required** environment variables. Please refer to the [examples](https://github.com/lovelaze/nebula-sync/tree/main/examples) for more detailed examples. + ### Docker CLI ```bash @@ -56,7 +56,6 @@ docker run --rm \ -e PRIMARY="http://ph1.example.com|password" \ -e REPLICAS="http://ph2.example.com|password" \ -e FULL_SYNC=true \ - -e RUN_GRAVITY=true \ ghcr.io/lovelaze/nebula-sync:latest ``` @@ -75,6 +74,8 @@ The following environment variables can be specified: | `REPLICAS`| n/a | `http://ph2.example.com\|password,http://ph3.example.com\|password` | Specifies the list of replica Pi-hole configurations | | `FULL_SYNC` | n/a | `true` | Specifies whether to perform a full synchronization | +> **Note:** If your Pi-hole instances are not using a password, you still need to include `|` but leave the password empty. e.g. `http://ph1.example.com|` + > **Note:** When `FULL_SYNC=true`, the system will perform a full Teleporter import/export from the primary Pi-hole to the replicas. This will synchronize all settings and configurations. > **Docker secrets:** `PRIMARY` and `REPLICAS` environment variables support Docker secrets when defined as `PRIMARY_FILE` and `REPLICAS_FILE`. See note regarding default user and Docker secrets example below. @@ -179,7 +180,7 @@ WEBHOOK_SYNC_FAILURE_HEADERS=Content-Type:application/json ## Notes / Known issues ### Default user of Docker container / Docker secrets example -By default, the Docker container runs as user `1001`. If you are using Docker secrets, the user that is running the container will need read permissions to the files that the Docker secrets reference. If the user does not have the right permissions you will receive an error `Failed to initialize service error="open /run/secrets/primary: permission denied"`. To avoid this error, either make sure to `chown 1001 ./your/secretfiles && chmod 400 ./your/secretfiles` or use the [`user` directive in Docker Compose](https://docs.docker.com/reference/compose-file/services/#user) to change the user that the container runs as to a user of your choice - and then make sure to update your secret files' ownership to that user. In the example [docker-compose-with-secrets.yml](examples/docker-compose-with-secrets.yml), user `1234` owns `./secrets/primary.txt` and `./secrets/replicas.txt` and both have `-r--------` permissions. +By default, the Docker container runs as user `1001`. If you are using Docker secrets, the user that is running the container will need read permissions to the files that the Docker secrets reference. If the user does not have the right permissions you will receive an error `Failed to initialize service error="open /run/secrets/primary: permission denied"`. To avoid this error, either make sure to `chown 1001 ./your/secretfiles && chmod 400 ./your/secretfiles` or use the [`user` directive in Docker Compose](https://docs.docker.com/reference/compose-file/services/#user) to change the user that the container runs as to a user of your choice - and then make sure to update your secret files' ownership to that user. In the example [docker-compose.yml](examples/docker-compose.yml), user `1234` owns `./secrets/primary.txt` and `./secrets/replicas.txt` and both have `-r--------` permissions. ### App passwords and authentication errors When using Pi-hole's app passwords ("Configure app password" in the Web interface / API settings page) with nebula-sync, you should enable the Pi-hole setting `webserver.api.app_sudo` on your `REPLICAS` servers or you may receive authentication errors. To configure this setting, perform one of the following: diff --git a/examples/.env.example b/examples/.env.example new file mode 100644 index 0000000..53f0870 --- /dev/null +++ b/examples/.env.example @@ -0,0 +1,141 @@ +################################ +# REQUIRED ENVIRONMENT VARIABLES +################################ + +PRIMARY=http://ph1.example.com|password +REPLICAS=http://ph2.example.com|password,http://ph3.example.com|password +# If you only have 1 replica, you may omit the comma. +# REPLICAS=http://ph2.example.com| +# If your Pi-hole instances are not using a password, you still need to include | but leave the password empty +# PRIMARY=http://ph1.example.com| +# REPLICAS=http://ph2.example.com|,http://ph3.example.com| + +# When FULL_SYNC=true, the system will perform a full Teleporter import/export from the primary Pi-hole to the replicas. +# This will synchronize all settings and configurations. +FULL_SYNC=true +# If you want to specify which settings to sync, you can set FULL_SYNC to false and then use the individual SYNC_* variables below. + +################################ +# OPTIONAL ENVIRONMENT VARIABLES +################################ + +# Specifies the cron schedule for synchronization +# CRON=0 * * * * + +# Specifies whether to run gravity after syncing +# RUN_GRAVITY=true + +# Specifies the timezone for logs and cron +# TZ=America/New_York + +# Specifies whether to skip TLS certificate verification on your Pi-hole instances +# CLIENT_SKIP_TLS_VERIFICATION=false + +# Specifies the delay between connection attempts +# CLIENT_RETRY_DELAY_SECONDS=1 + +# Specifies the timeout for HTTP requests +# CLIENT_TIMEOUT_SECONDS=20 + +########################### +# INDIVIDUAL SYNC VARIABLES +# Used when FULL_SYNC=false +########################### + +# Specifies whether to synchronize DNS settings +# SYNC_CONFIG_DNS=true + +# Specifies whether to synchronize DHCP settings +# SYNC_CONFIG_DHCP=true + +# Specifies whether to synchronize NTP settings +# SYNC_CONFIG_NTP=true + +# Specifies whether to synchronize resolver settings +# SYNC_CONFIG_RESOLVER=true + +# Specifies whether to synchronize database settings +# SYNC_CONFIG_DATABASE=true + +# Specifies whether to synchronize miscellaneous settings +# SYNC_CONFIG_MISC=true + +# Specifies whether to synchronize debug settings +# SYNC_CONFIG_DEBUG=true + +# Specifies whether to synchronize DHCP leases +# SYNC_GRAVITY_DHCP_LEASES=true + +# Specifies whether to synchronize groups +# SYNC_GRAVITY_GROUP=true + +# Specifies whether to synchronize ad lists +# SYNC_GRAVITY_AD_LIST=true + +# Specifies whether to synchronize ad lists by group +# SYNC_GRAVITY_AD_LIST_BY_GROUP=true + +# Specifies whether to synchronize domain lists +# SYNC_GRAVITY_DOMAIN_LIST=true + +# Specifies whether to synchronize domain lists by group +# SYNC_GRAVITY_DOMAIN_LIST_BY_GROUP=true + +# Specifies whether to synchronize clients +# SYNC_GRAVITY_CLIENT=true + +# Specifies whether to synchronize clients by group +# SYNC_GRAVITY_CLIENT_BY_GROUP=true + +# Specifies whether to synchronize DNS settings +# SYNC_CONFIG_DNS_INCLUDE=upstreams,interface +# SYNC_CONFIG_DNS_EXCLUDE=upstreams,interface + +# Specifies whether to synchronize DHCP settings +# SYNC_CONFIG_DHCP_INCLUDE=active,start +# SYNC_CONFIG_DHCP_EXCLUDE=active,start + +# Specifies whether to synchronize NTP settings +# SYNC_CONFIG_NTP_INCLUDE=ipv4,sync +# SYNC_CONFIG_NTP_EXCLUDE=ipv4,sync + +# Specifies whether to synchronize resolver settings +# SYNC_CONFIG_RESOLVER_INCLUDE=resolveIPv4,networkNames +# SYNC_CONFIG_RESOLVER_EXCLUDE=resolveIPv4,networkNames + +# Specifies whether to synchronize database settings +# SYNC_CONFIG_DATABASE_INCLUDE=DBimport,maxDBdays +# SYNC_CONFIG_DATABASE_EXCLUDE=DBimport,maxDBdays + +# Specifies whether to synchronize miscellaneous settings +# SYNC_CONFIG_MISC_INCLUDE=nice,delay_startup +# SYNC_CONFIG_MISC_EXCLUDE=nice,delay_startup + +# Specifies whether to synchronize debug settings +# SYNC_CONFIG_DEBUG_INCLUDE=database,networking +# SYNC_CONFIG_DEBUG_EXCLUDE=database,networking + +############################ +# OPTIONAL WEBHOOK VARIABLES +############################ + +# Nebula Sync can invoke webhooks depending if a sync succeeded or failed. +# URL is required for the webhook to trigger. +# Both success and failure webhooks use the same environment variable pattern. +# Webhooks have a timeout of 10 seconds. + +# Success webhook +# WEBHOOK_SYNC_SUCCESS_URL=https://www.example.com/webhook +# WEBHOOK_SYNC_SUCCESS_METHOD=POST +# WEBHOOK_SYNC_SUCCESS_BODY=this is my webhook body +# WEBHOOK_SYNC_SUCCESS_HEADERS=header1:foo,header2:bar + +# Failure webhook +# WEBHOOK_SYNC_FAILURE_URL=https://www.example.com/webhook +# WEBHOOK_SYNC_FAILURE_METHOD=POST +# WEBHOOK_SYNC_FAILURE_BODY=this is my webhook body +# WEBHOOK_SYNC_FAILURE_HEADERS=header1:foo,header2:bar + +# Specifies whether to skip TLS certificate verification for ALL webhooks +# WEBHOOK_CLIENT_SKIP_TLS_VERIFICATION=false + diff --git a/examples/.env_full b/examples/.env_full deleted file mode 100644 index 0a15b83..0000000 --- a/examples/.env_full +++ /dev/null @@ -1,5 +0,0 @@ -PRIMARY=http://ph1.example.com|password -REPLICAS=http://ph2.example.com|password -FULL_SYNC=true -RUN_GRAVITY=true -CRON=0 * * * * diff --git a/examples/.env_manual b/examples/.env_manual deleted file mode 100644 index dc009dd..0000000 --- a/examples/.env_manual +++ /dev/null @@ -1,22 +0,0 @@ -PRIMARY=http://ph1.example.com|password -REPLICAS=http://ph2.example.com|password -FULL_SYNC=false -RUN_GRAVITY=true -CRON=0 * * * * - -SYNC_CONFIG_DNS=true -SYNC_CONFIG_DHCP=true -SYNC_CONFIG_NTP=true -SYNC_CONFIG_RESOLVER=true -SYNC_CONFIG_DATABASE=true -SYNC_CONFIG_MISC=true -SYNC_CONFIG_DEBUG=true - -SYNC_GRAVITY_DHCP_LEASES=true -SYNC_GRAVITY_GROUP=true -SYNC_GRAVITY_AD_LIST=true -SYNC_GRAVITY_AD_LIST_BY_GROUP=true -SYNC_GRAVITY_DOMAIN_LIST=true -SYNC_GRAVITY_DOMAIN_LIST_BY_GROUP=true -SYNC_GRAVITY_CLIENT=true -SYNC_GRAVITY_CLIENT_BY_GROUP=true \ No newline at end of file diff --git a/examples/docker-compose-with-secrets.yml b/examples/docker-compose-with-secrets.yml deleted file mode 100644 index f02b38e..0000000 --- a/examples/docker-compose-with-secrets.yml +++ /dev/null @@ -1,43 +0,0 @@ -services: - nebula-sync: - image: ghcr.io/lovelaze/nebula-sync:latest - container_name: nebula-sync - restart: unless-stopped - # user directive specifying an arbitary uid to match ownership of Docker secrets files - user: 1234:1234 - secrets: - - primary - - replicas - environment: - - TZ=America/New_York - # Secrets are defined below - - PRIMARY_FILE=/run/secrets/primary - - REPLICAS_FILE=/run/secrets/replicas - # FULL_SYNC is false so that we can set specific EXCLUDES later - - FULL_SYNC=false - - RUN_GRAVITY=true - # Running every 15 minutes - - CRON=*/15 * * * * - # Giving additional time for slower devices - - CLIENT_RETRY_DELAY_SECONDS=15 - - CLIENT_TIMEOUT_SECONDS=45 - # Enabling selective syncs - - SYNC_GRAVITY_GROUP=true - - SYNC_GRAVITY_AD_LIST=true - - SYNC_GRAVITY_AD_LIST_BY_GROUP=true - - SYNC_GRAVITY_DOMAIN_LIST=true - - SYNC_GRAVITY_DOMAIN_LIST_BY_GROUP=true - - SYNC_GRAVITY_CLIENT=true - - SYNC_GRAVITY_CLIENT_BY_GROUP=true - - SYNC_CONFIG_NTP=true - - SYNC_CONFIG_DNS=true - # Excluding some device-specific portions of the DNS config - - SYNC_CONFIG_DNS_EXCLUDE=interface,reply.host.force4,reply.host.IPv4,reply.host.force6,reply.host.IPv6 - - -secrets: - primary: - # These files are owned by user 1234 and have -r----- permissions - file: ./secrets/primary.txt - replicas: - file: ./secrets/replicas.txt diff --git a/examples/docker-compose.yml b/examples/docker-compose.yml index 6278ed8..78b6534 100644 --- a/examples/docker-compose.yml +++ b/examples/docker-compose.yml @@ -3,8 +3,172 @@ services: nebula-sync: image: ghcr.io/lovelaze/nebula-sync:latest container_name: nebula-sync + restart: unless-stopped + + # You may use Docker secrets to store your primary and replica Pi-hole credentials. + # See `secrets` section below for required configuration. + # secrets: + # - primary + # - replicas + # You will also need to configure the following environment variables below: + # environment: + # - PRIMARY_FILE=/run/secrets/primary + # - REPLICAS_FILE=/run/secrets/replicas + + # You may specify a user and group to run the container as. + # If you are using Docker secrets files, this user must have the proper permissions to read the files. + # user: 1234:1234 + + # You may specify a volume to mount the configuration files to. + # volumes: + # - ./config:/app/config + environment: + ################################ + # REQUIRED ENVIRONMENT VARIABLES + ################################ + - PRIMARY=http://ph1.example.com|password - REPLICAS=http://ph2.example.com|password,http://ph3.example.com|password + # If you only have 1 replica, you may omit the comma. + # - REPLICAS=http://ph2.example.com|password + # If your Pi-hole instances are not using a password, you still need to include | but leave the password empty + # e.g. - PRIMARY=http://ph1.example.com| + # - REPLICAS=http://ph2.example.com|,http://ph3.example.com| + + # When FULL_SYNC=true, the system will perform a full Teleporter import/export from the primary Pi-hole to the replicas. + # This will synchronize all settings and configurations. - FULL_SYNC=true - - CRON=0 * * * * \ No newline at end of file + # If you want to specify which settings to sync, you can set FULL_SYNC to false and then use the individual SYNC_* variables below. + + ################################ + # OPTIONAL ENVIRONMENT VARIABLES + ################################ + + # Specifies the cron schedule for synchronization + # - CRON=0 * * * * + + # Specifies whether to run gravity after syncing + # - RUN_GRAVITY=true + + # Specifies the timezone for logs and cron + # - TZ=America/New_York + + # Specifies whether to skip TLS certificate verification on your Pi-hole instances + # - CLIENT_SKIP_TLS_VERIFICATION=false + + # Specifies the delay between connection attempts + # - CLIENT_RETRY_DELAY_SECONDS=1 + + # Specifies the timeout for HTTP requests + # - CLIENT_TIMEOUT_SECONDS=20 + + ########################### + # INDIVIDUAL SYNC VARIABLES + # Used when FULL_SYNC=false + ########################### + + # Specifies whether to synchronize DNS settings + # - SYNC_CONFIG_DNS=true + + # Specifies whether to synchronize DHCP settings + # - SYNC_CONFIG_DHCP=true + + # Specifies whether to synchronize NTP settings + # - SYNC_CONFIG_NTP=true + + # Specifies whether to synchronize resolver settings + # - SYNC_CONFIG_RESOLVER=true + + # Specifies whether to synchronize database settings + # - SYNC_CONFIG_DATABASE=true + + # Specifies whether to synchronize miscellaneous settings + # - SYNC_CONFIG_MISC=true + + # Specifies whether to synchronize debug settings + # - SYNC_CONFIG_DEBUG=true + + # Specifies whether to synchronize DHCP leases + # - SYNC_GRAVITY_DHCP_LEASES=true + + # Specifies whether to synchronize groups + # - SYNC_GRAVITY_GROUP=true + + # Specifies whether to synchronize ad lists + # - SYNC_GRAVITY_AD_LIST=true + + # Specifies whether to synchronize ad lists by group + # - SYNC_GRAVITY_AD_LIST_BY_GROUP=true + + # Specifies whether to synchronize domain lists + # - SYNC_GRAVITY_DOMAIN_LIST=true + + # Specifies whether to synchronize domain lists by group + # - SYNC_GRAVITY_DOMAIN_LIST_BY_GROUP=true + + # Specifies whether to synchronize clients + # - SYNC_GRAVITY_CLIENT=true + + # Specifies whether to synchronize clients by group + # - SYNC_GRAVITY_CLIENT_BY_GROUP=true + + # Specifies whether to synchronize DNS settings + # - SYNC_CONFIG_DNS_INCLUDE=upstreams,interface + # - SYNC_CONFIG_DNS_EXCLUDE=upstreams,interface + + # Specifies whether to synchronize DHCP settings + # - SYNC_CONFIG_DHCP_INCLUDE=active,start + # - SYNC_CONFIG_DHCP_EXCLUDE=active,start + + # Specifies whether to synchronize NTP settings + # - SYNC_CONFIG_NTP_INCLUDE=ipv4,sync + # - SYNC_CONFIG_NTP_EXCLUDE=ipv4,sync + + # Specifies whether to synchronize resolver settings + # - SYNC_CONFIG_RESOLVER_INCLUDE=resolveIPv4,networkNames + # - SYNC_CONFIG_RESOLVER_EXCLUDE=resolveIPv4,networkNames + + # Specifies whether to synchronize database settings + # - SYNC_CONFIG_DATABASE_INCLUDE=DBimport,maxDBdays + # - SYNC_CONFIG_DATABASE_EXCLUDE=DBimport,maxDBdays + + # Specifies whether to synchronize miscellaneous settings + # - SYNC_CONFIG_MISC_INCLUDE=nice,delay_startup + # - SYNC_CONFIG_MISC_EXCLUDE=nice,delay_startup + + # Specifies whether to synchronize debug settings + # - SYNC_CONFIG_DEBUG_INCLUDE=database,networking + # - SYNC_CONFIG_DEBUG_EXCLUDE=database,networking + + ############################ + # OPTIONAL WEBHOOK VARIABLES + ############################ + + # Nebula Sync can invoke webhooks depending if a sync succeeded or failed. + # URL is required for the webhook to trigger. + # Both success and failure webhooks use the same environment variable pattern. + # Webhooks have a timeout of 10 seconds. + + # Success webhook + # - WEBHOOK_SYNC_SUCCESS_URL=https://www.example.com/webhook + # - WEBHOOK_SYNC_SUCCESS_METHOD=POST + # - WEBHOOK_SYNC_SUCCESS_BODY=this is my webhook body + # - WEBHOOK_SYNC_SUCCESS_HEADERS=header1:foo,header2:bar + + # Failure webhook + # - WEBHOOK_SYNC_FAILURE_URL=https://www.example.com/webhook + # - WEBHOOK_SYNC_FAILURE_METHOD=POST + # - WEBHOOK_SYNC_FAILURE_BODY=this is my webhook body + # - WEBHOOK_SYNC_FAILURE_HEADERS=header1:foo,header2:bar + + # Specifies whether to skip TLS certificate verification for ALL webhooks + # - WEBHOOK_CLIENT_SKIP_TLS_VERIFICATION=false + +# You may use Docker secrets to store your primary and replica Pi-hole credentials. +# See `secrets` section above for corresponding configuration. +# secrets: +# primary: +# file: ./secrets/primary.txt +# replicas: +# file: ./secrets/replicas.txt