PMM HA v3 #632
Draft
PMM HA v3 #632
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Changed HTTP and HTTPS ports across configurations, updated HAProxy health check to use GET requests, and adjusted backend addresses. Refined Postgres HBA rules, updated security contexts, and switched image repository and tag for testing.
* adding users and password
* PMM-13857 * Make sure the PMM pods are running on different nodes * Comment out several PMM configuration options * * bump version * * pmm chart version bump * * fixing comments * fixing duplicate services * * remove blank lines * * add dependeces * * using latest image * * change name to pmm-ha * * revert changes in pmm/value.yaml * * testing if secrets and configmap added to workflow * * working on workflow to test-run the chart * * update chart version * * only pmm-ha chart * * moving all pmm-ha related jobs into a separate workflow yaml * * changing affinity * Update pod affinity settings in pmm-ha chart to use preferred scheduling instead of required * Update pmm-ha chart values to use dynamic release name and namespace for service endpoints * Refactor HAProxy configuration in pmm-ha chart: move config to values.yaml and remove hardcoded global service endpoints * Add global service endpoint definitions to pmm-ha chart values.yaml for dynamic service discovery * Add VMAuth configuration to victoria-metrics-cluster in pmm-ha chart, including dynamic service endpoints for vminsert and vmselect * Update pmm-ha chart values to switch PMM server image repository and tag, enhance pre-delete hook with kubectl installation, and correct DNS domain configuration in victoria-metrics-cluster * Update pmm chart version to 1.4.13 * Update pmm-ha chart version to 1.3.23 * Remove unused service port configurations for ClickHouse and Zookeeper in pmm-ha chart values.yaml * Update pmm-ha chart values to change PMM server image repository and tag, enhance pre-delete hook for PostgreSQL resource cleanup, and correct DNS configurations for PostgreSQL connections. * Update PostgreSQL connection settings in pmm-ha chart to use new database service name and ensure consistent naming for pg-database.fullname in helpers template. --------- Co-authored-by: Nurlan Moldomurov <[email protected]>
- Introduced an init container in the HAProxy configuration to wait for PMM instances to be ready before starting. - Added a new ConfigMap to store the init script that performs health checks on PMM instances. - Updated values.yaml to include the init container and additional volume configurations for the script.
- Updated the ConfigMap name for the HAProxy init container script to `pmm-ha-haproxy-init-script`. - Moved the init container script directly into the ConfigMap, enhancing clarity and maintainability. - Removed the old init container script definition from the helpers template.
PMM-14301 - Add init container for PMM readiness check in HAProxy configuration
Removed contributor spron-in from Chart.yaml
- Updated values.yaml to use template functions for PMM active IP and node ID. - Introduced new helper functions to generate PMM HA peer lists dynamically based on replicas count. - Modified HAProxy init configuration to reference the new dynamic service name. - Enhanced statefulset.yaml to utilize the new helper functions for PMM configuration.
- Disabled the default HAProxy configmap creation in values.yaml to allow for custom configurations. - Introduced a new ConfigMap template for HAProxy configuration, enhancing flexibility and maintainability.
- Removed hardcoded global service endpoints from values.yaml. - Updated the HAProxy configuration to utilize template functions for server definitions based on replicas count.
- Commented out several PMM configuration options for ClickHouse and PostgreSQL to enhance security and flexibility. - Adjusted database connection strings to use placeholders for credentials, improving maintainability.
- Updated the health check logic to use HTTP and HTTPS connection status codes for better accuracy. - Improved logging to provide detailed status messages for both HTTP and HTTPS checks. - Ensured that the script returns appropriate success or failure codes based on the connection checks.
- Modified server definitions in the HAProxy ConfigMap to utilize the service name from values.yaml, enhancing flexibility for service discovery. - Ensured both HTTP and HTTPS server configurations reflect the updated service name template.
…nets PMM-14364 Fix Patroni's pg_hba subnets
Co-authored-by: Alex Demidoff <[email protected]>
… dependency - Remove Bitnami ClickHouse subchart from charts/clickhouse/ - Add Altinity ClickHouse operator as dependency in Chart.yaml - Create ClickHouseInstallation template for operator - Add ClickHouse init scripts ConfigMap - Configure ClickHouse cluster with 3 replicas and pod anti-affinity - Update values.yaml with operator configuration - Set ClickHouse version to 23.3.21 - Configure PMM environment variables for ClickHouse connection Note: Operator approach has checksum issues - may need alternative approach
- Introduced a new Chart.lock file to manage HAProxy dependencies - Updated Chart.yaml to reflect the new HAProxy repository URL. - Enhanced values.yaml with detailed HAProxy configuration options, including TLS settings, monitoring, and security features. - Removed obsolete HAProxy chart files and templates to streamline the configuration management. - Added a new ConfigMap for HAProxy initialization and TLS secret management.
Co-authored-by: Alex Demidoff <[email protected]>
- Updated the introduction to specify operator-managed database backends. - Added a section on architecture, detailing the two-chart structure with dependencies. - Revised high availability features to reflect operator management for ClickHouse, VictoriaMetrics, and PostgreSQL clusters. - Clarified installation instructions for required Kubernetes operators, emphasizing the need for the `pmm-ha-dependencies` chart.
…ments PMM-14410 Various PMM HA improvements
PMM-14420 - Enhance PMM HA README with detailed features and installation instructions
- Added detailed instructions for connecting clients to PMM, including service endpoints and usage of service tokens for automation. - Updated values.yaml to include PMM integration for PostgreSQL monitoring, enabling automatic metric pushing and service account token management. - Refactored HAProxy ConfigMap to use a fixed service name for consistency across deployments. We can not use templates in values.yaml and pg operator needs the pmm hostname. - Introduced a new job template for creating service account tokens for PostgreSQL monitoring, streamlining the integration process.
- Changed the PMM readiness check loop to run indefinitely until PMM is ready, removing the maximum attempts limit. - Updated logging to reflect the current attempt number without the maximum attempts context. - Cleaned up unnecessary exit conditions related to the maximum attempts.
… access configuration - Added detailed instructions in README.md for configuring HAProxy service for external access, including LoadBalancer and NodePort options. - Updated values.yaml to include service type and annotations for HAProxy, providing examples for different cloud providers and MetalLB.
- Introduced a new inline scrape configuration for HAProxy metrics, allowing for dynamic scraping of HAProxy service endpoints. - Configured relabeling to filter and target the appropriate metrics path, enhancing monitoring capabilities.
PMM-14408 - scrape configuration for HAProxy metrics in vmagent.yaml
…ration - Added Prometheus metrics configuration for ClickHouse Keeper, including port and endpoint settings. - Introduced new inline scrape configurations for ClickHouse and ClickHouse Keeper metrics in vmagent.yaml, enabling dynamic scraping and relabeling for better monitoring. - Updated comments for clarity on the purpose of the configurations.
Enhance vmagent.yaml with additional scrape configurations for VMSelect, VMInsert, VMStorage, VMAuth, and VMAgent metrics. Updated inline comments for clarity on metrics collection.
PMM-14408 - Internal PostgreSQL monitoring
PMM-14408 - documentation and values.yaml for external access configuration
PMM-14406 - ClickHouse and ClickHouse Keeper metrics scraping configu…
- Added Prometheus metrics configuration for ClickHouse Keeper, including port and endpoint settings. - Updated vmagent.yaml to include new scrape configurations for ClickHouse and ClickHouse Keeper metrics, ensuring proper relabeling and targeting of metrics paths. - Improved inline comments for clarity on the purpose of each scrape configuration.
PMM-14407 - Scraping Victoriametrics
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.