diff --git a/docs/assets/fragments/haproxy-config.txt b/docs/assets/fragments/haproxy-config.txt new file mode 100644 index 0000000..5c93903 --- /dev/null +++ b/docs/assets/fragments/haproxy-config.txt @@ -0,0 +1,38 @@ +```default +global + maxconn 2048 + external-check + stats socket /var/run/haproxy.sock mode 600 expose-fd listeners level user +defaults + log global + mode tcp + retries 10 + timeout client 10000 + timeout connect 100500 + timeout server 10000 +frontend galera-in + bind *:3309 accept-proxy + bind *:3306 + mode tcp + option clitcpka + default_backend galera-nodes +frontend galera-admin-in + bind *:33062 + mode tcp + option clitcpka + default_backend galera-admin-nodes +frontend galera-replica-in + bind *:3307 + mode tcp + option clitcpka + default_backend galera-replica-nodes +frontend galera-mysqlx-in + bind *:33060 + mode tcp + option clitcpka + default_backend galera-mysqlx-nodes +frontend stats + bind *:8404 + mode http + http-request use-service prometheus-exporter if { path /metrics } +``` \ No newline at end of file diff --git a/docs/assets/fragments/proxysql-config.txt b/docs/assets/fragments/proxysql-config.txt new file mode 100644 index 0000000..202954e --- /dev/null +++ b/docs/assets/fragments/proxysql-config.txt @@ -0,0 +1,51 @@ +```default +datadir="/var/lib/proxysql" + +admin_variables = +{ + admin_credentials="proxyadmin:admin_password" + mysql_ifaces="0.0.0.0:6032" + refresh_interval=2000 + restapi_enabled=true + restapi_port=6070 + + cluster_username="proxyadmin" + cluster_password="admin_password" + cluster_check_interval_ms=200 + cluster_check_status_frequency=100 + cluster_mysql_query_rules_save_to_disk=true + cluster_mysql_servers_save_to_disk=true + cluster_mysql_users_save_to_disk=true + cluster_proxysql_servers_save_to_disk=true + cluster_mysql_query_rules_diffs_before_sync=1 + cluster_mysql_servers_diffs_before_sync=1 + cluster_mysql_users_diffs_before_sync=1 + cluster_proxysql_servers_diffs_before_sync=1 +} + +mysql_variables= +{ + monitor_password="monitor" + monitor_galera_healthcheck_interval=1000 + threads=2 + max_connections=2048 + default_query_delay=0 + default_query_timeout=10000 + poll_timeout=2000 + interfaces="0.0.0.0:3306" + default_schema="information_schema" + stacksize=1048576 + connect_timeout_server=10000 + monitor_history=60000 + monitor_connect_interval=20000 + monitor_ping_interval=10000 + ping_timeout_server=200 + commands_stats=true + sessions_sort=true + have_ssl=true + ssl_p2s_ca="/etc/proxysql/ssl-internal/ca.crt" + ssl_p2s_cert="/etc/proxysql/ssl-internal/tls.crt" + ssl_p2s_key="/etc/proxysql/ssl-internal/tls.key" + ssl_p2s_cipher="ECDHE-RSA-AES128-GCM-SHA256" +} +``` \ No newline at end of file diff --git a/docs/haproxy-conf.md b/docs/haproxy-conf.md index a359892..ee945bf 100644 --- a/docs/haproxy-conf.md +++ b/docs/haproxy-conf.md @@ -1,7 +1,7 @@ # Configuring Load Balancing with HAProxy -Percona Operator for MySQL based on Percona XtraDB Cluster provides a choice of two cluster components to -provide load balancing and proxy service: you can use either [HAProxy :octicons-link-external-16:](https://haproxy.org) or [ProxySQL :octicons-link-external-16:](https://proxysql.com/). +You can use either [HAProxy :octicons-link-external-16:](https://haproxy.org) or [ProxySQL :octicons-link-external-16:](https://proxysql.com/) for load balancing and proxy services. + You can control which one to use, if any, by enabling or disabling via the `haproxy.enabled` and `proxysql.enabled` options in the `deploy/cr.yaml` configuration file. @@ -25,27 +25,46 @@ $ kubectl patch pxc cluster1 --type=merge --patch '{ restart. Switching from HAProxy to ProxySQL is not possible, and if you need ProxySQL, this should be configured at cluster creation time. -The resulting HAProxy setup normally contains two services: +## HAProxy services + +The Operator creates two services for HAProxy: + +### `cluster1-haproxy` service + +The `cluster1-haproxy` service listens on the following ports: + +* `3306` is the default MySQL port. It is used by the mysql client, MySQL Connectors, and utilities such as mysqldump and mysqlpump + +* `3309` is the [proxy protocol :octicons-link-external-16:](https://www.haproxy.com/blog/haproxy/proxy-protocol/) port. Proxy protocol is used to store the client's IP address + +* `33062` is the port to connect to the MySQL Administrative Interface -* `cluster1-haproxy` service listening on ports 3306 (MySQL) and 3309 (the [proxy protocol :octicons-link-external-16:](https://www.haproxy.com/blog/haproxy/proxy-protocol/) useful for operations such as asynchronous calls). - This service is pointing to the number zero Percona XtraDB Cluster member - (`cluster1-pxc-0`) by default when this member is available. If a zero - member is not available, members are selected in descending order of their - numbers (e.g. `cluster1-pxc-2`, then `cluster1-pxc-1`, etc.). This service - can be used for both read and write load, or it can also be used just for - write load (single writer mode) in setups with split write and read loads. +* `33060` is the port for the [MySQLX protocol :octicons-link-external-16:](https://dev.mysql.com/doc/dev/mysql-server/8.4.3/PAGE_PROTOCOL.html). It is supported by clients such as MySQL Shell, MySQL Connectors and MySQL Router - [haproxy.enabled](operator.md#haproxyexposeprimaryenabled) +* `8404` is the port to connect to the [HAProxy statistics page :octicons-link-external-16:](https://www.haproxy.com/blog/exploring-the-haproxy-stats-page) + + The [haproxy.enabled](operator.md#haproxyexposeprimaryenabled) Custom Resource option enables or disables `cluster1-haproxy` service. -* `cluster1-haproxy-replicas` listening on port 3306 (MySQL). - This service selects Percona XtraDB Cluster members to serve queries following - the Round Robin load balancing algorithm. - It **should not be used for write requests**. +By default, the `cluster1-haproxy` service points to the number zero Percona XtraDB Cluster member (`cluster1-pxc-0`), when this member is available. If a zero member is not available, members are selected in descending order of their +numbers: `cluster1-pxc-2`, then `cluster1-pxc-1`. This service +can be used for both read and write load, or it can also be used just for +write load (single writer mode) in setups with split write and read loads. + +The [haproxy.exposePrimary.enabled](operator.md#haproxyexposeprimaryenabled) +Custom Resource option enables or disables the `cluster1-haproxy` service. + +### `cluster1-haproxy-replicas` service - [haproxy.exposeReplicas.enabled](operator.md#haproxyexposereplicasenabled) - Custom Resource option enables or disables `cluster1-haproxy-replicas` - service (on by default). +The `cluster1-haproxy-replicas` service listens on port 3306 (MySQL). + +This service selects Percona XtraDB Cluster members to serve queries following +the Round Robin load balancing algorithm. + +**Don't use it for write requests**. + +The [haproxy.exposeReplicas.enabled](operator.md#haproxyexposereplicasenabled) +Custom Resource option enables or disables `cluster1-haproxy-replicas` service (on by default). !!! note @@ -149,12 +168,29 @@ haproxy: mode tcp option clitcpka default_backend galera-nodes + + frontend galera-admin-in + bind *:33062 + mode tcp + option clitcpka + default_backend galera-admin-nodes + frontend galera-replica-in - bind *:3309 accept-proxy bind *:3307 mode tcp option clitcpka default_backend galera-replica-nodes + + frontend galera-mysqlx-in + bind *:33060 + mode tcp + option clitcpka + default_backend galera-mysqlx-nodes + + frontend stats + bind *:8404 + mode http + http-request use-service prometheus-exporter if { path /metrics } ``` ### Use a ConfigMap @@ -170,31 +206,7 @@ ConfigMap :octicons-link-external-16:](https://kubernetes.io/docs/tasks/configur For example, you define a `haproxy.cfg` configuration file with the following setting: -```default -global - maxconn 2048 - external-check - stats socket /var/run/haproxy.sock mode 600 expose-fd listeners level user -defaults - log global - mode tcp - retries 10 - timeout client 10000 - timeout connect 100500 - timeout server 10000 -frontend galera-in - bind *:3309 accept-proxy - bind *:3306 - mode tcp - option clitcpka - default_backend galera-nodes -frontend galera-replica-in - bind *:3309 accept-proxy - bind *:3307 - mode tcp - option clitcpka - default_backend galera-replica-nodes -``` +--8<-- "haproxy-config.txt" You can create a configmap from the `haproxy.cfg` file with the `kubectl create configmap` command. @@ -250,31 +262,7 @@ Actual options should be encoded with [Base64 :octicons-link-external-16:](https For example, let’s define a `haproxy.cfg` configuration file and put there options we used in the previous example: -```default -global - maxconn 2048 - external-check - stats socket /var/run/haproxy.sock mode 600 expose-fd listeners level user -defaults - log global - mode tcp - retries 10 - timeout client 10000 - timeout connect 100500 - timeout server 10000 -frontend galera-in - bind *:3309 accept-proxy - bind *:3306 - mode tcp - option clitcpka - default_backend galera-nodes -frontend galera-replica-in - bind *:3309 accept-proxy - bind *:3307 - mode tcp - option clitcpka - default_backend galera-replica-nodes -``` +--8<-- "haproxy-config.txt" You can get a Base64 encoded string from your options via the command line as follows: diff --git a/docs/proxysql-conf.md b/docs/proxysql-conf.md index a53e3f7..26c0038 100644 --- a/docs/proxysql-conf.md +++ b/docs/proxysql-conf.md @@ -1,22 +1,29 @@ # Configuring Load Balancing with ProxySQL -Percona Operator for MySQL based on Percona XtraDB Cluster provides a choice of two cluster components to -provide load balancing and proxy service: you can use either [HAProxy :octicons-link-external-16:](https://haproxy.org) or [ProxySQL :octicons-link-external-16:](https://proxysql.com/). -You can choose which one to use, if any, by enabling or disabling via the +You can use either [HAProxy :octicons-link-external-16:](https://haproxy.org) or [ProxySQL :octicons-link-external-16:](https://proxysql.com/) for load balancing and proxy services. + +You can control which one to use: enable or disable the `haproxy.enabled` and `proxysql.enabled` options in the `deploy/cr.yaml` configuration file. !!! warning - You can enable ProxySQL only at cluster creation time. Otherwise you will be - able to use HAProxy only, and the switch from HAProxy to ProxySQL is not + You can enable ProxySQL only when you create a cluster. For a running cluster you can enable only HAProxy. Also note, if you have already enabled HAProxy, the switch from it to ProxySQL is not possible. -The resulting setup will use the number zero Percona XtraDB Cluster member -(`cluster1-pxc-0` by default) as writer. +## `cluster1-proxysql` service + +The `cluster1-proxysql` service listens on the following ports: + +* `3306` is the default MySQL port. It is used by the mysql client, MySQL Connectors, and utilities such as mysqldump and mysqlpump +* `33062` is the port to connect to the MySQL Administrative Interface +* `6070` is the port to connect to the built-in Prometheus exporter to gather ProxySQL statistics and manage the ProxySQL observability stack + +The `cluster1-proxysql` service uses the number zero Percona XtraDB Cluster member +(`cluster1-pxc-0` by default) as the writer. [proxysql.expose.enabled](operator.md#proxysqlexposeenabled) Custom Resource -option enables or disables the apropriate `cluster1-proxysql` service. +option enables or disables the `cluster1-proxysql` service. !!! note @@ -25,7 +32,7 @@ option enables or disables the apropriate `cluster1-proxysql` service. (e.g. to use on the tenant network), add the following [annotation](annotations.md) in the Custom Resource metadata section of the `deploy/cr.yaml`: - ```yaml + ```yaml apiVersion: pxc.percona.com/v1 kind: PerconaXtraDBCluster metadata: @@ -84,6 +91,8 @@ proxysql: admin_credentials="proxyadmin:admin_password" mysql_ifaces="0.0.0.0:6032" refresh_interval=2000 + restapi_enabled=true + restapi_port=6070 cluster_username="proxyadmin" cluster_password="admin_password" @@ -139,55 +148,7 @@ ConfigMap :octicons-link-external-16:](https://kubernetes.io/docs/tasks/configur For example, you define a `proxysql.cnf` configuration file with the following setting: -```default -datadir="/var/lib/proxysql" - -admin_variables = -{ - admin_credentials="proxyadmin:admin_password" - mysql_ifaces="0.0.0.0:6032" - refresh_interval=2000 - - cluster_username="proxyadmin" - cluster_password="admin_password" - cluster_check_interval_ms=200 - cluster_check_status_frequency=100 - cluster_mysql_query_rules_save_to_disk=true - cluster_mysql_servers_save_to_disk=true - cluster_mysql_users_save_to_disk=true - cluster_proxysql_servers_save_to_disk=true - cluster_mysql_query_rules_diffs_before_sync=1 - cluster_mysql_servers_diffs_before_sync=1 - cluster_mysql_users_diffs_before_sync=1 - cluster_proxysql_servers_diffs_before_sync=1 -} - -mysql_variables= -{ - monitor_password="monitor" - monitor_galera_healthcheck_interval=1000 - threads=2 - max_connections=2048 - default_query_delay=0 - default_query_timeout=10000 - poll_timeout=2000 - interfaces="0.0.0.0:3306" - default_schema="information_schema" - stacksize=1048576 - connect_timeout_server=10000 - monitor_history=60000 - monitor_connect_interval=20000 - monitor_ping_interval=10000 - ping_timeout_server=200 - commands_stats=true - sessions_sort=true - have_ssl=true - ssl_p2s_ca="/etc/proxysql/ssl-internal/ca.crt" - ssl_p2s_cert="/etc/proxysql/ssl-internal/tls.crt" - ssl_p2s_key="/etc/proxysql/ssl-internal/tls.key" - ssl_p2s_cipher="ECDHE-RSA-AES128-GCM-SHA256" -} -``` +--8<-- "proxysql-config.txt" You can create a configmap from the `proxysql.cnf` file with the `kubectl create configmap` command. @@ -243,55 +204,8 @@ Actual options should be encoded with [Base64 :octicons-link-external-16:](https For example, let’s define a `proxysql.cnf` configuration file and put there options we used in the previous example: -```default -datadir="/var/lib/proxysql" - -admin_variables = -{ - admin_credentials="proxyadmin:admin_password" - mysql_ifaces="0.0.0.0:6032" - refresh_interval=2000 - - cluster_username="proxyadmin" - cluster_password="admin_password" - cluster_check_interval_ms=200 - cluster_check_status_frequency=100 - cluster_mysql_query_rules_save_to_disk=true - cluster_mysql_servers_save_to_disk=true - cluster_mysql_users_save_to_disk=true - cluster_proxysql_servers_save_to_disk=true - cluster_mysql_query_rules_diffs_before_sync=1 - cluster_mysql_servers_diffs_before_sync=1 - cluster_mysql_users_diffs_before_sync=1 - cluster_proxysql_servers_diffs_before_sync=1 -} - -mysql_variables= -{ - monitor_password="monitor" - monitor_galera_healthcheck_interval=1000 - threads=2 - max_connections=2048 - default_query_delay=0 - default_query_timeout=10000 - poll_timeout=2000 - interfaces="0.0.0.0:3306" - default_schema="information_schema" - stacksize=1048576 - connect_timeout_server=10000 - monitor_history=60000 - monitor_connect_interval=20000 - monitor_ping_interval=10000 - ping_timeout_server=200 - commands_stats=true - sessions_sort=true - have_ssl=true - ssl_p2s_ca="/etc/proxysql/ssl-internal/ca.crt" - ssl_p2s_cert="/etc/proxysql/ssl-internal/tls.crt" - ssl_p2s_key="/etc/proxysql/ssl-internal/tls.key" - ssl_p2s_cipher="ECDHE-RSA-AES128-GCM-SHA256" -} -``` +--8<-- "proxysql-config.txt" + You can get a Base64 encoded string from your options via the command line as follows: