Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support providing ZDM configuration as YAML file #71

Merged
merged 4 commits into from
Jul 8, 2024
Merged

Support providing ZDM configuration as YAML file #71

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bc6e085
Support providing ZDM configuration as YAML file
lukasz-antoniak Jul 4, 2024
c2d9c84
Include credential parameters in YAML file
lukasz-antoniak Jul 4, 2024
74ca965
Validate ZDM Proxy configuration mode
lukasz-antoniak Jul 8, 2024
7c8671a
Apply review comments
lukasz-antoniak Jul 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 35 additions & 6 deletions ansible/deploy_zdm_proxy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,29 +209,39 @@
- name: Create ZDM proxy configuration env file
hosts: proxies
vars_files:
- vars/zdm_proxy_container_config.yml
- vars/zdm_proxy_cluster_config.yml
- vars/zdm_proxy_core_config.yml
- vars/zdm_proxy_advanced_config.yml
- vars/zdm_proxy_custom_tls_config.yml
- vars/zdm_playbook_internal_config.yml

tasks:
- name: Applying default ZDM proxy config mode
when: zdm_proxy_config_mode is undefined
set_fact:
zdm_proxy_config_mode: "env_vars"
- name: Verify proxy container configuration
assert:
that: zdm_proxy_config_mode == "env_vars" or zdm_proxy_config_mode == "conf_file"
success_msg: "ZDM proxy container correctly configured"
fail_msg: "Invalid value of 'zdm_proxy_config_mode' parameter"
Copy link
Collaborator

@alicel alicel Jul 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good and should be clearer for users. i would suggest two very small changes to improve clarity.

(1) make naming more consistent. the parameter is called zdm_proxy_config_mode but on of the possible values is conf_file. this small inconsistency (conf vs config) could be confusing. i would stick to config for both the parameter name and its potential value, as we already use config in other variable names.

(2) explicitly list the two supported modes in the error message, so the user can easily see if they accidentally passed an invalid value (env-vars, env_var, conf-file, or similar things). for example, if you opted for the config naming, you could have something like: Invalid value for the 'zdm_proxy_config_mode' parameter: accepted values are 'env_vars' or 'config_file'

- name: Create configuration fragment directory
file:
path: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_config_fragments_dir_name }}"
state: directory
- name: Generate env var file from template for immutable configuration
- name: Generate configuration file from template for immutable configuration
template:
src: "zdm_proxy_immutable_config.j2"
src: "{{ 'zdm_proxy_immutable_config_env_vars.j2' if zdm_proxy_config_mode == 'env_vars' else 'zdm_proxy_immutable_config_file.j2' }}"
dest: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_config_fragments_dir_name }}/{{ zdm_proxy_immutable_config_fragment_file_name }}"
- name: Generate env var file from template for mutable configuration
- name: Generate configuration file from template for mutable configuration
template:
src: "zdm_proxy_mutable_config.j2"
src: "{{ 'zdm_proxy_mutable_config_env_vars.j2' if zdm_proxy_config_mode == 'env_vars' else 'zdm_proxy_mutable_config_file.j2' }}"
dest: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_config_fragments_dir_name }}/{{ zdm_proxy_mutable_config_fragment_file_name }}"
- name: Merge the immutable and mutable configuration files into a single one
assemble:
src: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_config_fragments_dir_name }}"
dest: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_environment_config_file_name }}"
dest: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_environment_config_file_name if zdm_proxy_config_mode == 'env_vars' else (zdm_proxy_shared_assets_dir_name + '/' + zdm_proxy_config_file_name) }}"

- name: Install ZDM Proxy
hosts: proxies
Expand All @@ -251,7 +261,8 @@
vars:
ansible_python_interpreter: python3

- name: Create ZDM proxy container
- name: Create ZDM proxy container with env vars
when: "zdm_proxy_config_mode == 'env_vars'"
docker_container:
name: "{{ zdm_proxy_container_name }}"
image: "docker.io/{{ zdm_proxy_image }}"
Expand All @@ -272,6 +283,24 @@
vars:
ansible_python_interpreter: python3

- name: Create ZDM proxy container with configuration file
when: "zdm_proxy_config_mode == 'conf_file'"
docker_container:
name: "{{ zdm_proxy_container_name }}"
image: "docker.io/{{ zdm_proxy_image }}"
command:
- "--config={{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/{{zdm_proxy_config_file_name}}"
mounts:
- source: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}"
target: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}"
type: bind
network_mode: host
restart_policy: unless-stopped
restart: yes
state: started
vars:
ansible_python_interpreter: python3

- name: Wait for this ZDM proxy to come up
uri:
url: "http://{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}:{{ metrics_port }}/health/readiness"
Expand Down
41 changes: 34 additions & 7 deletions ansible/rolling_update_zdm_proxy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,20 +18,30 @@
- name: Create ZDM proxy configuration env file based on the latest configuration
hosts: proxies
vars_files:
- vars/zdm_proxy_container_config.yml
- vars/zdm_proxy_cluster_config.yml
- vars/zdm_proxy_core_config.yml
- vars/zdm_proxy_advanced_config.yml
- vars/zdm_playbook_internal_config.yml

tasks:
- name: Generate env var file from template for mutable configuration
- name: Applying default ZDM proxy config mode
when: zdm_proxy_config_mode is undefined
set_fact:
zdm_proxy_config_mode: "env_vars"
- name: Verify proxy container configuration
assert:
that: zdm_proxy_config_mode == "env_vars" or zdm_proxy_config_mode == "conf_file"
success_msg: "ZDM proxy container correctly configured"
fail_msg: "Invalid value of 'zdm_proxy_config_mode' parameter"
- name: Generate configuration file from template for mutable configuration
template:
src: "zdm_proxy_mutable_config.j2"
src: "{{ 'zdm_proxy_mutable_config_env_vars.j2' if zdm_proxy_config_mode == 'env_vars' else 'zdm_proxy_mutable_config_file.j2' }}"
dest: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_config_fragments_dir_name }}/{{ zdm_proxy_mutable_config_fragment_file_name }}"
- name: Merge the immutable and mutable configuration files into a single one
assemble:
src: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_config_fragments_dir_name }}"
dest: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_environment_config_file_name }}"
dest: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_environment_config_file_name if zdm_proxy_config_mode == 'env_vars' else (zdm_proxy_shared_assets_dir_name + '/' + zdm_proxy_config_file_name) }}"

- name: Configure and restart each ZDM proxy in a rolling fashion
hosts: proxies
Expand All @@ -45,7 +55,8 @@
- vars/zdm_proxy_core_config.yml # this inclusion is for backward compatibility and will be removed in the future

tasks:
- name: Create ZDM proxy container with updated configuration
- name: Create ZDM proxy container with updated env vars
when: "zdm_proxy_config_mode == 'env_vars'"
docker_container:
name: "{{ zdm_proxy_container_name }}"
image: "{{ zdm_proxy_image }}"
Expand All @@ -65,6 +76,25 @@
state: started
vars:
ansible_python_interpreter: python3

- name: Create ZDM proxy container with updated configuration file
when: "zdm_proxy_config_mode == 'conf_file'"
docker_container:
name: "{{ zdm_proxy_container_name }}"
image: "{{ zdm_proxy_image }}"
command:
- "--config={{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/{{zdm_proxy_config_file_name}}"
mounts:
- source: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}"
target: "{{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}"
type: bind
network_mode: host
restart_policy: unless-stopped
restart: yes
state: started
vars:
ansible_python_interpreter: python3

- name: Wait for this ZDM proxy to come up
uri:
url: "http://{{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}:{{ metrics_port }}/health/readiness"
Expand All @@ -76,6 +106,3 @@
- pause:
prompt: "Pause for {{ pause_between_restarts_in_seconds }} seconds after restarting this ZDM proxy instance"
seconds: "{{ pause_between_restarts_in_seconds }}"



0 ...e/templates/zdm_proxy_immutable_config.j2 → ...es/zdm_proxy_immutable_config_env_vars.j2
View file
Open in desktop
File renamed without changes.
72 changes: 72 additions & 0 deletions ansible/templates/zdm_proxy_immutable_config_file.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
#jinja2: lstrip_blocks: "True", trim_blocks: "True"

{% set zdm_proxy_address_list = [] %}
{% for host in groups['proxies'] %}
{{ zdm_proxy_address_list.append(host) }}
{% endfor %}
proxy_topology_index: {{ groups['proxies'].index(hostvars[inventory_hostname]['ansible_default_ipv4']['address']) }}
proxy_topology_addresses: {{ zdm_proxy_address_list|join(',') }}

{% if ( origin_contact_points is defined ) %}
origin_contact_points: {{ origin_contact_points }}
{% if ( origin_port is defined ) %}
origin_port: {{ origin_port }}
{% endif %}
{% elif ( ( origin_scb_downloaded is defined and origin_scb_downloaded == 'success' ) or ( origin_scb_provided is defined and origin_scb_provided == 'success' ) )%}
origin_secure_connect_bundle_path: {{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/origin_scb.zip
{% endif %}
{% if ( origin_local_datacenter is defined ) %}
origin_local_datacenter: {{ origin_local_datacenter }}
{% endif %}

{% if ( target_contact_points is defined ) %}
target_contact_points: {{ target_contact_points }}
{% if ( target_port is defined ) %}
target_port: {{ target_port }}
{% endif %}
{% elif ( target_scb_downloaded is defined and target_scb_downloaded == 'success' ) or ( target_scb_provided is defined and target_scb_provided == 'success' ) %}
target_secure_connect_bundle_path: {{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/target_scb.zip
{% endif %}
{% if ( target_local_datacenter is defined ) %}
target_local_datacenter: {{ target_local_datacenter }}
{% endif %}

proxy_listen_address: {{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}
proxy_listen_port: {{ zdm_proxy_listen_port }}

metrics_address: {{ hostvars[inventory_hostname]['ansible_default_ipv4']['address'] }}
metrics_port: {{ metrics_port }}

{% if ( origin_tls_user_dir_path is defined and origin_tls_server_ca_filename is defined ) %}
origin_tls_server_ca_path: {{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/{{ origin_tls_dest_dir_name }}/{{ origin_tls_server_ca_filename }}
{% endif %}
{% if ( origin_tls_user_dir_path is defined and origin_tls_client_cert_filename is defined ) %}
origin_tls_client_cert_path: {{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/{{ origin_tls_dest_dir_name }}/{{ origin_tls_client_cert_filename }}
{% endif %}
{% if ( origin_tls_user_dir_path is defined and origin_tls_client_key_filename is defined ) %}
origin_tls_client_key_path: {{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/{{ origin_tls_dest_dir_name }}/{{ origin_tls_client_key_filename }}
{% endif %}

{% if ( target_tls_user_dir_path is defined and target_tls_server_ca_filename is defined ) %}
target_tls_server_ca_path: {{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/{{ target_tls_dest_dir_name }}/{{ target_tls_server_ca_filename }}
{% endif %}
{% if ( target_tls_user_dir_path is defined and target_tls_client_cert_filename is defined ) %}
target_tls_client_cert_path: {{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/{{ target_tls_dest_dir_name }}/{{ target_tls_client_cert_filename }}
{% endif %}
{% if ( target_tls_user_dir_path is defined and target_tls_client_key_filename is defined ) %}
target_tls_client_key_path: {{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/{{ target_tls_dest_dir_name }}/{{ target_tls_client_key_filename }}
{% endif %}

{% if ( zdm_proxy_tls_user_dir_path is defined and zdm_proxy_tls_ca_filename is defined ) %}
proxy_tls_ca_path: {{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/{{ zdm_proxy_tls_dest_dir_name }}/{{ zdm_proxy_tls_ca_filename }}
{% endif %}
{% if ( zdm_proxy_tls_user_dir_path is defined and zdm_proxy_tls_cert_filename is defined ) %}
proxy_tls_cert_path: {{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/{{ zdm_proxy_tls_dest_dir_name }}/{{ zdm_proxy_tls_cert_filename }}
{% endif %}
{% if ( zdm_proxy_tls_user_dir_path is defined and zdm_proxy_tls_key_filename is defined ) %}
proxy_tls_key_path: {{ zdm_proxy_home_dir }}/{{ zdm_proxy_shared_assets_dir_name }}/{{ zdm_proxy_tls_dest_dir_name }}/{{ zdm_proxy_tls_key_filename }}
{% endif %}
{% if ( zdm_proxy_tls_require_client_auth is defined ) %}
proxy_tls_require_client_auth: {{ zdm_proxy_tls_require_client_auth }}
{% endif %}

0 ...ble/templates/zdm_proxy_mutable_config.j2 → ...ates/zdm_proxy_mutable_config_env_vars.j2
View file
Open in desktop
File renamed without changes.
58 changes: 58 additions & 0 deletions ansible/templates/zdm_proxy_mutable_config_file.j2
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#jinja2: lstrip_blocks: "True", trim_blocks: "True"

{% if ( primary_cluster is defined ) %}
primary_cluster: {{ primary_cluster }}
{% endif %}

{% if ( read_mode is defined ) %}
read_mode: {{ read_mode }}
{% endif %}

{% if ( log_level is defined ) %}
log_level: {{ log_level }}
{% endif %}

origin_username: {{ origin_username | default('') }}
origin_password: {{ origin_password | default('') }}
target_username: {{ target_username | default('') }}
target_password: {{ target_password | default('') }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where are these settings in the env vars mode?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh they are included as env vars in the actual runbook yml, in that case we need to make sure these env vars are not set if the config mode is file config

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is already taken care of.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nevermind, I went back and read it again and I understand now.


{% if ( zdm_proxy_max_clients_connections is defined ) %}
proxy_max_client_connections: {{ zdm_proxy_max_clients_connections }}
{% endif %}

{% if ( zdm_proxy_request_timeout_ms is defined ) %}
proxy_request_timeout_ms: {{ zdm_proxy_request_timeout_ms }}
{% endif %}
{% if ( origin_connection_timeout_ms is defined ) %}
origin_connection_timeout_ms: {{ origin_connection_timeout_ms }}
{% endif %}
{% if ( target_connection_timeout_ms is defined ) %}
target_connection_timeout_ms: {{ target_connection_timeout_ms }}
{% endif %}
{% if ( async_handshake_timeout_ms is defined ) %}
async_handshake_timeout_ms: {{ async_handshake_timeout_ms }}
{% endif %}
{% if ( heartbeat_interval_ms is defined ) %}
heartbeat_interval_ms: {{ heartbeat_interval_ms }}
{% endif %}
{% if ( zdm_proxy_max_stream_ids is defined ) %}
proxy_max_stream_ids: {{ zdm_proxy_max_stream_ids }}
{% endif %}

{% if ( metrics_enabled is defined ) %}
metrics_enabled: {{ metrics_enabled }}
{% endif %}

{% if ( system_queries_mode is defined ) %}
system_queries_mode: {{ system_queries_mode }}
{% endif %}

{% if ( replace_cql_functions is defined ) %}
replace_cql_functions: {{ replace_cql_functions }}
{% endif %}

{% if ( forward_client_credentials_to_origin is defined ) %}
forward_client_credentials_to_origin: {{ forward_client_credentials_to_origin }}
{% endif %}

1 change: 1 addition & 0 deletions ansible/vars/zdm_playbook_internal_config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ zdm_proxy_config_fragments_dir_name: zdm_proxy_config_fragments
zdm_proxy_mutable_config_fragment_file_name: zdm_proxy_mutable_config.env
zdm_proxy_immutable_config_fragment_file_name: zdm_proxy_immutable_config.env
zdm_proxy_environment_config_file_name: zdm_proxy_config.env
zdm_proxy_config_file_name: zdm_proxy_config.yml

origin_scb_file_name: origin_scb
target_scb_file_name: target_scb
Expand Down
6 changes: 5 additions & 1 deletion ansible/vars/zdm_proxy_container_config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,8 @@
zdm_proxy_image: datastax/zdm-proxy:2.x

create_containers: 1
zdm_proxy_container_name: zdm-proxy-container
zdm_proxy_container_name: zdm-proxy-container

# Defines how configuration shall be passed to ZDM proxy.
# Allowed values are "env_vars" and "conf_file".
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there some pre-check we can do on this value before running a runbook? I don't know much about ansible to know how feasible this would be. Maybe a task that runs before Create ZDM proxy container ... ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could do below before creating configuration files, but in the end we do not validate other parameters (e.g. contact points):

    - name: Verify proxy container configuration
      assert:
        that: zdm_proxy_config_mode == "env_vars" or zdm_proxy_config_mode == "conf_file"
        success_msg: "ZDM proxy container correctly configured"
        fail_msg: "Invalid value of 'zdm_proxy_config_mode' parameter"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Contact points will be validated by the proxy, automation just passes them through. This setting is an automation specific setting.

Copy link
Collaborator

@alicel alicel Jul 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The general guideline with Ansible is typically to rely on the playbook letting you know that it failed and why. Ansible is pretty good at that, it usually returns meaningful errors and it is idempotent, so if you fix the issue and re-run a partially executed playbook it will skip the steps that it has already done and pick up any change, re-executing only whatever is affected by the change and then anything that was not executed at all of course.

However we have to see how this works for us in this particular case.

I would say that there are two things to check:

  • if the parameter is not set at all, does the execution default to env_vars? which i think is what we wanted in order to have backward compatibility
  • if the parameter is explicitly set to an incorrect value (e.g. env_var or config_file), does the playbook fail in a visible and clear way?

If we see that it doesn't explicitly fail but silently executes incorrectly, or if the failure is not clear, validating this particular parameter may be appropriate.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added application of default value and explicit value check. In other cases, expressions at various places are getting more and more complex (we always need to check if value is undefined).

zdm_proxy_config_mode: env_vars
Loading