Skip to content

Commit 8ff4b83

Browse files
tgadievpavelpikta
authored andcommitted
Zookeeper debug fix (#18)
* refactoring: simplify the logic of adding new systems * add zookeeper installation from choco * update by ansible development kit * fix vars * fix windows choco test * added part of tarball installation * fixed lint * fixed tarboll install * fixed service issue * fixed choco install * fixed tempate issues * added gitlab ci tests * fixed gitlab ci tests * fixed aws win playbook * changed gitlab ci tests and deafaults * changed gitlab ci tests * added gitlab ci tests aws direct * fix gitlab ci tests * revert changes * fix windows installation * fix java installation * fix java installation * refactoring: update README, fix quotes * fix install path * debug current tasks * fix tests * debug 2 * fix err * add verify for linux * changes to verify * debian package debug * debug zookeeper service * linux tests * full test * fix readme remove debug * suggested changes * remove debug and minor fixes * Test single quotes for Win Signed-off-by: Timur Gadiev <[email protected]> * Fix lint and service Signed-off-by: Timur Gadiev <[email protected]> * Enable Linux test Signed-off-by: Timur Gadiev <[email protected]> * Add backslash escaping Signed-off-by: Timur Gadiev <[email protected]> * Add Amazon2 tests Signed-off-by: Timur Gadiev <[email protected]> * some fixes, debug zk_dir * add handler for Windows, full test * enable all win tests, fix readme * Test separate prepare Signed-off-by: Timur Gadiev <[email protected]> * Test combined Linux with Windows nodes Signed-off-by: Timur Gadiev <[email protected]> * Fix molecule inventory Signed-off-by: Timur Gadiev <[email protected]> * Change to ansible 2.7 Signed-off-by: Timur Gadiev <[email protected]> * Test windows Signed-off-by: Timur Gadiev <[email protected]> * Test win_template Signed-off-by: Timur Gadiev <[email protected]> * Fix win_template configs Signed-off-by: Timur Gadiev <[email protected]> * Test force_config Signed-off-by: Timur Gadiev <[email protected]> * Change indent Signed-off-by: Timur Gadiev <[email protected]> * Test without zk_config Signed-off-by: Timur Gadiev <[email protected]> Signed-off-by: Timur Gadiev <[email protected]> * Test two plays Signed-off-by: Timur Gadiev <[email protected]> * Fix lint Signed-off-by: Timur Gadiev <[email protected]> * Test debug service Signed-off-by: Timur Gadiev <[email protected]> * Test wait for Signed-off-by: Timur Gadiev <[email protected]> * Test win wait for Signed-off-by: Timur Gadiev <[email protected]> * Increase timeout Signed-off-by: Timur Gadiev <[email protected]> * Debug pre tasks Signed-off-by: Timur Gadiev <[email protected]> * Debug ansible_facts Signed-off-by: Timur Gadiev <[email protected]> * Test service start retry Signed-off-by: Timur Gadiev <[email protected]> * Add debug log dir Signed-off-by: Timur Gadiev <[email protected]> * Add ignore_errors: true Signed-off-by: Timur Gadiev <[email protected]> Signed-off-by: Timur Gadiev <[email protected]> * Fix config dynamic Signed-off-by: Timur Gadiev <[email protected]> * Test with docker Signed-off-by: Timur Gadiev <[email protected]> * Enable all tests Signed-off-by: Timur Gadiev <[email protected]> * Fix prepare path Signed-off-by: Timur Gadiev <[email protected]> * Add debug files Signed-off-by: Timur Gadiev <[email protected]> * Fix lint Signed-off-by: Timur Gadiev <[email protected]> * Fix set_fact Signed-off-by: Timur Gadiev <[email protected]> * Add package version Signed-off-by: Timur Gadiev <[email protected]> * Update docs and tests Signed-off-by: Timur Gadiev <[email protected]>
1 parent cd473fa commit 8ff4b83

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

49 files changed

+802
-195
lines changed

.cookiecutter.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
---
2+
default_context:
3+
role_name: zookeeper

.github/pull_request_template.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# Pull Request Template
2+
3+
## Description
4+
5+
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
6+
7+
Fixes # (issue)
8+
9+
## Type of change
10+
11+
Please delete options that are not relevant.
12+
13+
- [ ] Bug fix (non-breaking change which fixes an issue)
14+
- [ ] New feature (non-breaking change which adds functionality)
15+
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
16+
- [ ] This change requires a documentation update
17+
18+
## Reviews
19+
20+
Please identify developer to review this change
21+
22+
- [ ] @developer
23+
24+
## Checklist:
25+
26+
- [ ] I have performed a self-review of my own code
27+
- [ ] I have made corresponding changes to the documentation
28+
- [ ] My changes generate no new warnings
29+
- [ ] I have added tests that prove my fix is effective or that my feature works
30+
- [ ] New and existing tests pass with my changes

.gitlab-ci.yml

Lines changed: 21 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,29 @@
11
---
2-
# Run all tests
32
stages:
43
- lint
54
- deployment test
65

76
before_script:
87
- rm -rf molecule/resources/provisioning
98
- ansible --version
9+
- ansible-lint --version
1010
- molecule --version
1111
- git clone https://github.com/lean-delivery/ansible-molecule-drivers.git molecule/resources/provisioning
1212

1313
variables:
1414
GET_SOURCES_ATTEMPTS: "5"
1515

16-
Ansible lint:
16+
Lint:
1717
stage: lint
1818
before_script:
1919
- git clone https://github.com/lean-delivery/ansible-lint-rules.git ~/ansible-lint-rules
2020
script:
21+
- yamllint . -c .yamllint
2122
- ansible-lint . -c .ansible-lint
2223
after_script:
2324
- rm -rf ~/ansible-lint-rules
2425
tags:
25-
- aws
26+
- lint
2627

2728
Docker zookeeper:
2829
stage: deployment test
@@ -40,7 +41,23 @@ AWS zookeeper:
4041
tags:
4142
- aws
4243

43-
EPC zookeeper:
44+
AZURE zookeeper:
45+
stage: deployment test
46+
script:
47+
- molecule test -s cloud-azure-direct
48+
tags:
49+
- azure
50+
51+
.AWS Windows zookeeper:
52+
variables:
53+
AWS_REGION: us-east-1
54+
stage: deployment test
55+
script:
56+
- molecule test -s cloud-aws-windows
57+
tags:
58+
- aws
59+
60+
.EPC zookeeper:
4461
variables:
4562
EPC_REGION: EPAM-BY2
4663
stage: deployment test

.travis.yml

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,26 @@
11
---
2-
dist: xenial
2+
dist: bionic
33
sudo: required
44

55
language: python
66
python:
7-
- "2.7"
7+
- "3.7"
88

99
services:
1010
- docker
1111

1212
before_install:
13+
- git clone https://github.com/lean-delivery/ansible-molecule-drivers.git molecule/resources/provisioning
1314
- git clone https://github.com/lean-delivery/ansible-lint-rules.git ~/ansible-lint-rules
1415

1516
install:
16-
- pip install --upgrade ansible==2.7.* docker-py molecule==2.20 pyOpenSSL PyYAML pytest
17-
- pip install git+https://github.com/ansible/ansible-lint.git
17+
- pip3 install --upgrade ansible==2.8.* docker molecule==2.22
18+
- pip3 install git+https://github.com/ansible/ansible-lint.git
1819
- ansible --version
1920
- ansible-lint --version
2021

2122
script:
23+
- yamllint . -c .yamllint
2224
- ansible-lint . -c .ansible-lint
2325
- molecule test -s default
2426

LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -198,4 +198,4 @@ Apache License
198198
distributed under the License is distributed on an "AS IS" BASIS,
199199
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200200
See the License for the specific language governing permissions and
201-
limitations under the License.
201+
limitations under the License.

README.md

Lines changed: 55 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -8,135 +8,130 @@ Zookeeper installation
88
![Ansible](https://img.shields.io/badge/dynamic/json.svg?label=min_ansible_version&url=https%3A%2F%2Fgalaxy.ansible.com%2Fapi%2Fv1%2Froles%2F36578%2F&query=$.min_ansible_version)
99

1010
This role:
11-
- Installs ZooKeeper
11+
- Installs ZooKeeper:
12+
- Linux:
13+
* tarball
14+
* package
15+
- Windows:
16+
* tarball
17+
* chocolatey
1218
- Configures it as a single node or a cluster
1319

1420
Requirements
1521
------------
1622

1723
- Minimal Version of the ansible for installation: 2.7
1824
Supported OS:
19-
- CentOS
25+
- CentOS
2026
7
21-
- Ubuntu
27+
- Ubuntu
2228
- 16.04
2329
- 18.04
30+
- Amazon Linux 2
2431
- Debian
2532
- 9
33+
- Windows
34+
- 2016
35+
- 2019
2636

2737
Role Variables
2838
--------------
2939

30-
- `zk_version` - version of the package
40+
- `zk_config` - custom zookeeper configuration parameters
41+
default: `[]`
3142

32-
default: `3.4.14`
33-
34-
- `zk_url` - download url
35-
36-
default: `http://www.us.apache.org/dist/zookeeper/zookeeper-{{zk_version}}/zookeeper-{{zk_version}}.tar.gz` for versions < 3.5
43+
- `zk_version` - version of the package. Tarball installation can use `current` or `stable` value of this variable which links to related zookeeper versions. Installation from repositories do not require this version to be specified.
44+
default: `current`
3745

46+
- `zk_url` - download url
47+
default: `http://www.us.apache.org/dist/zookeeper/zookeeper-{{zk_version}}/zookeeper-{{zk_version}}.tar.gz` for versions < 3.5
3848
default: `http://www.us.apache.org/dist/zookeeper/zookeeper-{{zk_version}}/apache-zookeeper-{{zk_version}}-bin.tar.gz` for versions 3.5.x
3949

40-
- `zk_tarball_installation` - installation from tarball(or repository)
41-
50+
- `zk_tarball_installation` - installation from tarball. Linux support install via repository and tarball. Windows support install via chocolatey and tarball
4251
default: `true`
4352

44-
- `zk_user` - OS user name for zookeeper
45-
53+
- `zk_user` - OS user name for zookeeper
4654
default: `zookeeper`
4755

48-
- `zk_group` - OS user group name for zookeeper
49-
56+
- `zk_group` - OS user group name for zookeeper
5057
default: `zookeeper`
5158

52-
- `zk_debian_apt_repositories` - path to apt repository
53-
59+
- `zk_debian_apt_repositories` - path to apt repository
5460
default: `"deb http://us-east1.gce.archive.ubuntu.com/ubuntu/ bionic universe"`
5561

56-
- `zk_redhat_yum_repositories` - path to yum repository
57-
62+
- `zk_redhat_yum_repositories` - path to yum repository
5863
default: `- https://archive.cloudera.com/cdh5/one-click-install/redhat/7/x86_64/cloudera-cdh-5-0.x86_64.rpm`
5964

60-
- `zk_client_port` - zookeeper port
61-
65+
- `zk_client_port` - zookeeper port
6266
default: `2181`
6367

64-
- `zk_init_limit` - amount of time in ticks to allow followers to connect and sync to a leader
65-
68+
- `zk_init_limit` - amount of time in ticks to allow followers to connect and sync to a leader
6669
default: `5`
6770

68-
- `zk_sync_limit` - amount of time in ticks to allow followers to sync with ZooKeeper
69-
71+
- `zk_sync_limit` - amount of time in ticks to allow followers to sync with ZooKeeper
7072
default: `2`
7173

72-
- `zk_tick_time` - it is used to regulate heartbeats, and timeouts. For example, the minimum session timeout will be two ticks
73-
74+
- `zk_tick_time` - it is used to regulate heartbeats, and timeouts. For example, the minimum session timeout will be two ticks
7475
default: `2000`
7576

76-
- `zk_autopurge_purgeInterval` - the time interval in hours for which the purge task has to be triggered
77-
77+
- `zk_autopurge_purgeInterval` - the time interval in hours for which the purge task has to be triggered
7878
default: `0`
7979

80-
- `zk_autopurge_snapRetainCount` - when enabled, ZooKeeper auto purge feature retains the autopurge.snapRetainCount most recent snapshots and the corresponding transaction logs in the dataDir and dataLogDir respectively and deletes the rest
81-
80+
- `zk_autopurge_snapRetainCount` - when enabled, ZooKeeper auto purge feature retains the autopurge.snapRetainCount most recent snapshots and the corresponding transaction logs in the dataDir and dataLogDir respectively and deletes the rest
8281
default: `10`
8382

84-
- `zk_data_dir` - libraries directory
85-
86-
default: `/var/lib/zookeeper`
87-
88-
- `zk_log_dir` - logs directory
83+
- `zk_data_dir` - libraries directory
84+
default: `'{{ zk_default_data_dir }}'`
8985

90-
default: `/var/log/zookeeper`
86+
- `zk_log_dir` - logs directory
87+
default: `{{ zk_default_log_dir }}`
9188

92-
- `zk_dir` - zookeeper direcotry
89+
- `zk_dir` - zookeeper directory
90+
default: `{{ zk_default_dir }}`
9391

94-
default: `"{{ zk_tarball_installation | ternary('/opt/zookeeper-' + zk_version, '/usr/lib/zookeeper') }}"`
95-
96-
- `zk_force_myid` - to reset id
92+
- `zk_choko_basedir` - zookeeper base directory when installing from chocolatey
93+
default: `C:\zookeeper`
9794

95+
- `zk_force_myid` - to reset id
9896
default: `true`
9997

100-
- `zk_force_config` - to rewrite config files
101-
98+
- `zk_force_config` - to rewrite config files
10299
default: `true`
103100

104-
- `zk_tarball_dir` - place where you download tarball
105-
106-
default: `/opt/src`
107-
108-
- `zk_rolling_log_file_max_size` - zookeeper log file size
101+
- `zk_tarball_dir` - place where you download tarball
102+
default: `{{ zk_default_tarball_dir }}`
109103

104+
- `zk_rolling_log_file_max_size` - zookeeper log file size
110105
default: `10MB`
111106

112-
- `zk_max_rolling_log_file_count`- zookeeper log file count
113-
107+
- `zk_max_rolling_log_file_count`- zookeeper log file count
114108
default: `10`
115109

116-
- `zk_inventory_group` - zookeeper inventory group name
117-
110+
- `zk_inventory_group` - ansible inventory group name for hosts running zookeeper
118111
default: `zookeeper`
119112

120-
- `zk_service_name` - zookeeper service name
113+
- `zk_address` - zookeeper node communication address. Usual usage assume this to be an address of local interface inside trusted network.
114+
default: `'{{ ansible_host }}'`
121115

116+
- `zk_service_name` - zookeeper service name
122117
default: `zookeeper`
123118

124-
- `zk_service_start` - to start zookeeper service in the end of role/Playbook
125-
119+
- `zk_service_start` - to start zookeeper service in the end of role/Playbook
126120
default: `true`
127121

128-
- `zk_service_autostart` - Add zookeeper service to automatically start.
129-
122+
- `zk_service_autostart` - add zookeeper service to automatically start
130123
default: `true`
131124

132-
- `zk_reconfig_enabled` - This option is introduced such that the reconfiguration feature can be completely disabled and any attempts to reconfigure a cluster through reconfig API with or without authentication will fail by default
125+
- `zk_standalone_enabled` - with this setting it is possible to start a ZooKeeper ensemble containing a single participant and to dynamically grow it by adding more servers
126+
default: `true`
133127

128+
- `zk_reconfig_enabled` - This option is introduced such that the reconfiguration feature can be completely disabled and any attempts to reconfigure a cluster through reconfig API with or without authentication will fail by default
134129
default: `true`
135130

136131
Dependencies
137132
------------
138133

139-
https://github.com/lean-delivery/ansible-role-java
134+
ZooKeeper runs in Java, release 1.6 or greater (JDK 6 or greater) is required. Java can be installed by corresponding Lean Delivery ansible role as an option: [![Galaxy](https://img.shields.io/badge/galaxy-lean__delivery.java-blue.svg)](https://galaxy.ansible.com/lean_delivery/java)
140135

141136
Example Inventory
142137
----------------
@@ -166,7 +161,7 @@ Example Playbook
166161
167162
License
168163
-------
169-
Apache
164+
Apache 2.0 [![License](https://img.shields.io/badge/license-Apache-green.svg?style=flat)](https://raw.githubusercontent.com/lean-delivery/ansible-role-zookeeper/master/LICENSE)
170165
171166
Author Information
172167
------------------

defaults/main.yml

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
zk_config: []
3-
zk_version: 3.4.14
4-
zk_url: '{{ default_zk_url }}'
3+
zk_version: '{{ zk_default_version }}'
4+
zk_url: '{{ zk_default_url }}'
55
zk_tarball_installation: true
66
zk_user: zookeeper
77
zk_group: zookeeper
@@ -14,17 +14,19 @@ zk_tick_time: 2000
1414
zk_autopurge_purgeInterval: 0
1515
zk_autopurge_snapRetainCount: 10
1616
# 3.5.0 and later
17-
zk_reconfig_enabled: 'true'
18-
zk_standalone_enabled: 'true'
19-
zk_data_dir: /var/lib/zookeeper
20-
zk_log_dir: /var/log/zookeeper
21-
zk_dir: '{{ zk_tarball_installation | ternary("/opt/zookeeper-" + zk_version, "/usr/lib/zookeeper") }}'
17+
zk_reconfig_enabled: true
18+
zk_standalone_enabled: true
19+
zk_data_dir: '{{ zk_default_data_dir }}'
20+
zk_log_dir: '{{ zk_default_log_dir }}'
21+
zk_dir: '{{ zk_default_dir }}'
22+
zk_choko_basedir: C:\zookeeper
2223
zk_force_myid: true
2324
zk_force_config: true
24-
zk_tarball_dir: /opt/src
25+
zk_tarball_dir: '{{ zk_default_tarball_dir }}'
2526
zk_rolling_log_file_max_size: 10MB
2627
zk_max_rolling_log_file_count: 10
2728
zk_inventory_group: zookeeper
29+
zk_address: '{{ ansible_host }}'
2830

2931
zk_service_name: zookeeper
3032
zk_service_start: true

handlers/main.yml

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,15 @@
11
---
22
- name: restart Zookeeper
33
systemd:
4-
name: zookeeper
4+
name: '{{ zk_service_name }}'
55
state: restarted
6+
when:
7+
- zk_service_start | bool
68
become: true
9+
10+
- name: restart Zookeeper for Windows
11+
win_service:
12+
name: '{{ zk_service_name }}'
13+
state: restarted
714
when:
8-
- zk_service_start
15+
- zk_service_start | bool

0 commit comments

Comments
 (0)