Skip to content

Commit 55f29e7

Browse files
jordipiquesellesjordipiqueselles
committed
Refactor the operators_test and report test coverage
The operators_test was a badly designed test suite, since it said it was just testing the operators but in fact it was also testing the process of parsing them, since it's much easier to specify operations using the GenericWebhookConfig CRD. The problem was that the CRD will evolve and the generic-k8s-webhook should support (and test) both old and new versions of the CRD. The refactor of the tests makes easier to keep test coverage for all the versions that are supported. Apart from that, we're introducing pytest-cov to write a test coverage report after executing all the unittests.
1 parent bde1671 commit 55f29e7

8 files changed

+684
-639
lines changed

‎Makefile

+3-2
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
SRC_DIR := ./generic_k8s_webhook
22
TEST_DIR := ./tests
33
SRC_FILES := $(shell find $(SRC_DIR) -type f -name '*.py')
4-
TEST_FILES := $(shell find $(TEST_DIR) -type f -name '*.py')
4+
TEST_FILES := $(shell find $(TEST_DIR) -type f -name '*.py' -o -name '*.yaml')
55

66
out/install-deps.stamp: pyproject.toml poetry.lock
77
poetry install
@@ -29,7 +29,8 @@ format: build
2929

3030
.PHONY: unittests
3131
unittests: build
32-
poetry run pytest tests --timeout 15
32+
poetry run pytest tests
33+
poetry run coverage html
3334

3435
.PHONY: check-pyproject
3536
check-pyproject:

‎docs/contributor-guide.md

+6
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,12 @@ We use [pytest](https://docs.pytest.org/en/7.3.x/) to test the functionality of
6565
make unittests
6666
```
6767

68+
The previous command will generate a test coverage report. You can open it on your browser by typing:
69+
70+
```bash
71+
open htmlcov/index.html
72+
```
73+
6874
### Docker build
6975

7076
The last phase of our testing suite is building the docker container that has our app installed in it.

‎poetry.lock

+426-325
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎pyproject.toml

+4
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ pytest-timeout = "^2.1.0"
2222
isort = "^5.12.0"
2323
black = "^23.3.0"
2424
pylint = "^2.17.4"
25+
pytest-cov = "^5.0.0"
2526

2627
[tool.isort]
2728
line_length = 120
@@ -40,6 +41,9 @@ too-few-public-methods, \
4041
fixme, \
4142
"""
4243

44+
[tool.pytest.ini_options]
45+
addopts = "-v --timeout=15 --cov=generic_k8s_webhook"
46+
4347
[build-system]
4448
requires = ["poetry-core>=1.0.0"]
4549
build-backend = "poetry.core.masonry.api"

‎tests/conditions_test.py

+52
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
import os
2+
3+
import pytest
4+
import yaml
5+
6+
from generic_k8s_webhook.config_parser.entrypoint import GenericWebhookConfigManifest
7+
8+
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
9+
CONDITIONS_YAML = os.path.join(SCRIPT_DIR, "conditions_test.yaml")
10+
11+
12+
def _expand_schemas(schemas_subsets: dict[str, list[str]], list_schemas: list[str]) -> list[str]:
13+
final_schemas = set()
14+
for schema in list_schemas:
15+
final_schemas.add(schema)
16+
for schemas_superset in schemas_subsets.get(schema, []):
17+
final_schemas.add(schemas_superset)
18+
return sorted(list(final_schemas))
19+
20+
21+
def _parse_tests() -> list[tuple]:
22+
with open(CONDITIONS_YAML, "r") as f:
23+
raw_tests = yaml.safe_load(f)
24+
25+
parsed_tests = []
26+
for test_suite in raw_tests["test_suites"]:
27+
for test in test_suite["tests"]:
28+
for schema in _expand_schemas(raw_tests["schemas_subsets"], test["schemas"]):
29+
for i, case in enumerate(test["cases"]):
30+
parsed_tests.append(
31+
(
32+
f"{test_suite['name']}_{i}", # name
33+
schema, # schema
34+
case["condition"], # condition
35+
case.get("context", [{}]), # context
36+
case["expected_result"], # expected_result
37+
)
38+
)
39+
return parsed_tests
40+
41+
42+
@pytest.mark.parametrize(("name", "schema", "condition", "context", "expected_result"), _parse_tests())
43+
def test_all(name, schema, condition, context, expected_result):
44+
raw_config = {
45+
"apiVersion": f"generic-webhook/{schema}",
46+
"kind": "GenericWebhookConfig",
47+
"webhooks": [{"name": "test-webhook", "path": "test-path", "actions": [{"condition": condition}]}],
48+
}
49+
gwcm = GenericWebhookConfigManifest(raw_config)
50+
action = gwcm.list_webhook_config[0].list_actions[0]
51+
result = action.condition.get_value(context)
52+
assert result == expected_result

‎tests/conditions_test.yaml

+193
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,193 @@
1+
schemas_subsets:
2+
# v1alpha1 is a subset of the features from v1beta1
3+
# For this reason, any test that uses v1alpha1 will also be executed for v1beta1
4+
v1alpha1:
5+
- v1beta1
6+
7+
# Each suite stresses a specific operator or set of operators used to specify conditions
8+
# Each test specifies the schema used to parse the condition. Notice that the same test
9+
# will be executed by all the schemas that are a superset of the one specified in the test.
10+
# A test is composed by different cases. Each case defines the actual condition to be tested,
11+
# some context (optional) and the expected result.
12+
test_suites:
13+
- name: AND
14+
tests:
15+
- schemas: [v1alpha1]
16+
cases:
17+
- condition:
18+
and:
19+
- const: true
20+
- const: true
21+
expected_result: true
22+
- condition:
23+
and:
24+
- const: true
25+
- const: false
26+
expected_result: false
27+
- condition:
28+
and:
29+
- const: false
30+
expected_result: false
31+
- condition:
32+
and: []
33+
expected_result: true
34+
- name: OR
35+
tests:
36+
- schemas: [v1alpha1]
37+
cases:
38+
- condition:
39+
or:
40+
- const: false
41+
- const: false
42+
expected_result: false
43+
- condition:
44+
or:
45+
- const: true
46+
- const: false
47+
expected_result: true
48+
- condition:
49+
or:
50+
- const: true
51+
expected_result: true
52+
- name: NOT
53+
tests:
54+
- schemas: [v1alpha1]
55+
cases:
56+
- condition:
57+
not:
58+
const: true
59+
expected_result: false
60+
- condition:
61+
not:
62+
const: false
63+
expected_result: true
64+
- name: EQUAL
65+
tests:
66+
- schemas: [v1alpha1]
67+
cases:
68+
- condition:
69+
equal:
70+
- const: 1
71+
expected_result: true
72+
- condition:
73+
equal:
74+
- const: 1
75+
- const: 2
76+
expected_result: false
77+
- condition:
78+
equal:
79+
- const: 2
80+
- const: 2
81+
expected_result: true
82+
- name: SUM
83+
tests:
84+
- schemas: [v1alpha1]
85+
cases:
86+
- condition:
87+
sum:
88+
- const: 1
89+
- const: 2
90+
- const: 3
91+
expected_result: 6
92+
- condition:
93+
sum:
94+
- const: 2
95+
expected_result: 2
96+
- name: GET_VALUE
97+
tests:
98+
- schemas: [v1alpha1]
99+
cases:
100+
# Retrieve value from last context
101+
- condition:
102+
getValue: .name
103+
context:
104+
- metadata:
105+
name: foo
106+
spec: {}
107+
- name: bar
108+
expected_result: bar
109+
# Retrieve value from first context
110+
- condition:
111+
getValue: $.metadata.name
112+
context:
113+
- metadata:
114+
name: foo
115+
spec: {}
116+
- name: bar
117+
expected_result: foo
118+
- name: FOR_EACH
119+
tests:
120+
- schemas: [v1alpha1]
121+
cases:
122+
# Iterate over a constant list of elements and sum 10 to each
123+
- condition:
124+
forEach:
125+
elements:
126+
const: [1, 2]
127+
op:
128+
sum:
129+
- const: 10
130+
- getValue: "."
131+
expected_result: [11, 12]
132+
# Iterate over a list defined in the yaml file and sum 1 to each
133+
- condition:
134+
forEach:
135+
elements:
136+
getValue: .containers
137+
op:
138+
sum:
139+
- const: 1
140+
- getValue: .maxCPU
141+
context:
142+
- containers:
143+
- maxCPU: 1
144+
- maxCPU: 2
145+
expected_result: [2, 3]
146+
- name: CONTAIN
147+
tests:
148+
- schemas: [v1alpha1]
149+
cases:
150+
- condition:
151+
contain:
152+
elements:
153+
getValue: .containers
154+
value:
155+
const: { maxCPU: 2 }
156+
context:
157+
- containers:
158+
- maxCPU: 1
159+
- maxCPU: 2
160+
expected_result: true
161+
- condition:
162+
contain:
163+
elements:
164+
getValue: .containers
165+
value:
166+
const: { maxCPU: 4 }
167+
context:
168+
- containers:
169+
- maxCPU: 1
170+
- maxCPU: 2
171+
expected_result: false
172+
- name: RAW_STR_EXPR
173+
tests:
174+
- schemas: [v1beta1]
175+
cases:
176+
- condition: "2 * (3 + 4 / 2) - 1"
177+
expected_result: 9
178+
- condition: "2*(3+4/2)-1"
179+
expected_result: 9
180+
- condition: "8/4/2"
181+
expected_result: 1
182+
- condition: "1 == 1 && 1 != 0 && 0 <= 0 && 0 < 1 && 1 > 0 && 1 >= 1 && true"
183+
expected_result: true
184+
- condition: "1 != 1 || 1 == 0 || 0 < 0 || 0 >= 1 || 1 <= 0 || 1 < 1 || false"
185+
expected_result: false
186+
- condition: '"foo" == "foo" && "foo" != "bar"'
187+
expected_result: true
188+
- condition: ".containers.0.maxCPU + 1 == .containers.1.maxCPU"
189+
context:
190+
- containers:
191+
- maxCPU: 1
192+
- maxCPU: 2
193+
expected_result: true

0 commit comments

Comments
 (0)