Skip to content

Commit 9e2d4b7

Browse files
FrawlessFrawless
authored
Use parse-comment from main branch (#11861)
Signed-off-by: Jakub Stejskal <xstejs24@gmail.com>
1 parent 2c74682 commit 9e2d4b7

File tree

8 files changed

+214
-39
lines changed

8 files changed

+214
-39
lines changed

‎.github/actions/systemtests/parse-comment/action.yml‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,14 @@ runs:
7777
// Auto-run smoke tests on PR workflow changes
7878
shouldRun = true;
7979
} else if (ev === 'issue_comment' && context.payload?.issue?.pull_request) {
80-
shouldRun = /^\s*\/gha\s+run\b/i.test(body || '');
80+
// For issue comments on PRs, check if PR is closed first
81+
const issueState = context.payload?.issue?.state;
82+
if (issueState === 'closed') {
83+
core.info(`PR #${context.payload.issue.number} is closed, skipping execution`);
84+
shouldRun = false;
85+
} else {
86+
shouldRun = /^\s*\/gha\s+run\b/i.test(body || '');
87+
}
8188
}
8289
8390
core.setOutput('shouldRun', String(shouldRun));
@@ -134,8 +141,8 @@ runs:
134141
};
135142
136143
const DEFAULTS = {
137-
pipelines: 'regression,upgrade',
138-
profiles: 'operators,operands,brokers-and-security,azp_kraft_upgrade,azp_kafka_upgrade',
144+
pipelines: '',
145+
profiles: '',
139146
featureGate: '+DummyFeatureGate',
140147
rbacScope: 'CLUSTER',
141148
installType: 'yaml',

‎.github/docs/README.md‎

Lines changed: 43 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Building Strimzi
1+
# Build & Test Strimzi in GitHub Actions
22
The following lines put together basic information how is Strimzi built and tested via GitHub Actions.
33
Currently, our main build system still lives on Azure Pipelines and running builds and tests on GHA is in experimental phase.
44
All the steps mentioned bellow re-use scripts from our [azure](../../.azure) directory and thus all the steps are more or less the same on both systems.
@@ -69,6 +69,7 @@ System tests execution actions:
6969
- [set-defaults](../actions/utils/set-defaults)
7070
- [log-variables](../actions/utils/log-variables)
7171
- [parse-comment](../actions/systemtests/parse-comment)
72+
- [determine-ref](../actions/determine-ref)
7273

7374
Utils actions:
7475
- [check-permissions](../actions/utils/check-permissions)
@@ -87,7 +88,8 @@ The actions also had to be put together into workflow as we have in Azure:
8788
- `operator-release pipeline` -> `operator-release workflow`
8889

8990
## Running system tests
90-
With GitHub Actions we are now able to propagate a specific parameters to our e2e jobs which allow us to just run a subset of tests for example or run it against different kubernetes version etc.
91+
With GitHub Actions we are now able to propagate a specific parameters to our e2e jobs.
92+
That allow us to run a subset of tests for example or run it against different kubernetes version etc.
9193

9294
### Triggers
9395
There are two options how the workflows could be triggered - manually via GitHub UI, via issue comment.
@@ -97,44 +99,57 @@ Comment for triggering the workflow has to starts with string `/gha run` and the
9799
The whole script that parse the trigger even is part of [parse-comment](../actions/systemtests/parse-comment) action.
98100
Currently, we have these parameters that can be passed through the comment:
99101

100-
| Name | Info | Default |
101-
|-------------------------------|------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------|
102-
| pipeline | Name of the pipeline from [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) that wil be executed | regression,upgrade,performance |
103-
| profile | Testing profile from pom that will be executed | regression,upgrade,performance |
104-
| agent | Agent that will be used for a specific pipeline (see list of runners in Strimzi org config for more info) | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
105-
| strimzi_feature_gates | Which Strimzi Feature Gates will be used | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
106-
| strimzi_rbac_scope | RBAC scope for Strimzi | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
107-
| cluster_operator_install_type | How Strimzi will be installed during the tests | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
108-
| parallel | Number of tests that will be executed in parallel | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
109-
| architecture | Which architecture will be used (should match with agent arch) | Value set in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) |
110-
| groups | Which Junit5 groups will be executed | all |
111-
| tests | Which Junit5 tests will be executed | all |
112-
| kubeVersion | Used Kubernetes version as part of Kind/Minikube setup | The one set as default in setup scripts |
113-
| kafkaVersion | Which Kafka version will be used in the tests | Default one from STs config |
114-
115-
Note that parameters are passed only to a `custom` pipeline except `kafkaVersion` and `kubeVersion` that are used for all jobs.
116-
117-
For trigger via GitHub UI you can specify `releaseVersion`, `kafkaVersion`, and `profile`.
118-
The first two parameters are working in the same manner as in Azure.
119-
`profile` is used to filter out all pipelines defined in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) that contains one of the profile specified in comma separated list in `profile` parameter.
102+
| Name | Info | Default |
103+
|-------------------------------|------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------|
104+
| pipeline | Name of the pipeline from [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) that wil be executed | regression,upgrade |
105+
| profile | Testing profile from [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) that will be executed, it has to be defined in pom file | smoke |
106+
| agent | Agent that will be used for a specific pipeline (see list of runners in Strimzi org config for more info) | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
107+
| strimzi_feature_gates | Which Strimzi Feature Gates will be used | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
108+
| strimzi_rbac_scope | RBAC scope for Strimzi | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
109+
| cluster_operator_install_type | How Strimzi will be installed during the tests | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
110+
| parallel | Number of tests that will be executed in parallel | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
111+
| architecture | Which architecture will be used (should match with agent arch) | Value set in [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) |
112+
| groups | Which Junit5 groups will be executed | all |
113+
| tests | Which Junit5 tests will be executed | all |
114+
| kubeVersion | Used Kubernetes version as part of Kind/Minikube setup | The one set as default in setup scripts |
115+
| kafkaVersion | Which Kafka version will be used in the tests | Default one from STs config |
116+
117+
The process of parameter usage is as follows:
118+
- `pipeline` has the highest priority. If `pipeline` is defined, the jobs will be loaded with data from [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) that match specific _pipeline_.
119+
- `profile` has the second-highest priority. If `profile` is defined, the jobs will be loaded with data from [pipelines.yaml](../actions/generate-matrix/pipelines.yaml) that match specific _profile_.
120+
- `agent` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
121+
- `strimzi_feature_gates` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
122+
- `strimzi_rbac_scope` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
123+
- `cluster_operator_install_type` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
124+
- `parallel` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
125+
- `architecture` is used only for a `custom` pipeline when either `pipeline` or `profile` are not specified via comment.
126+
- `groups` are passed directly to `mvn` command. It should be used mostly only with `custom` pipeline as otherwise a specific `pipeline` could have the `groups` excluded.
127+
- `tests` are passed directly to `mvn` command. It should be used mostly only with `custom` pipeline as otherwise a specific `pipeline` could have the `tests` excluded.
128+
- `kubeVersion` is used for setup `kind` for every running jobs.
129+
- `kafkaVersion` is used for every running job.
120130

121131
### Matrix generation
122-
Once the event is parsed the mechanism will decide whether Strimzi should be built or already existing images will be used (release rc for example).
123-
After that job matrix will be generated [run-system-tests](../workflows/run-system-tests.yml) workflow will be invoked for each job defined in the matrix.
132+
Once the event is parsed the mechanism will decide whether Strimzi should be built or already existing images will be used (release RC for example).
133+
After that job matrix will be generated and [run-system-tests](../workflows/run-system-tests.yml) workflow will be invoked for each job defined in the matrix.
124134

125-
Matrix is generated by action [generate-matrix](../actions/systemtests/generate-matrix) either for custom pipeline mentioned above or for pipelines defined in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml).
135+
Matrix is generated by action [generate-matrix](../actions/systemtests/generate-matrix) either by filtering pipelines/profiles defined in [pipelines.yaml](../actions/systemtests/generate-matrix/pipelines.yaml) or by creating a custom pipeline from passed params.
126136

127137
### Workflow depiction
128138
![sts-workflow.png](sts-workflow.png)
129139

130140
## Security
131141
To achieve a better security we agreed to not allow everyone to trigger the system tests pipelines in the similar way as we do in Azure.
132142
The main part in access restriction is [check-permissions](../actions/utils/check-permissions) action that basically check access rights of the user who triggered the pipeline.
133-
If the user has at least write access or is part of Strimzi org, then it will allow the execution.
143+
If the user has at least write access or is part of Strimzi org, then it will allow the execution.
134144
The action is used only from main branch which removes the possibility that a random user will simply change the actions/workflow and the use the resources on his own.
135145
Also keep in mind that workflow for system tests will be used from a specific branch (main/release-X) for `workflow_dispatch` and `issue_comment` events.
136146

137147
Each first-time contributor has to be approved by one of the maintainers which should avoid us to hit some unexpected changes to the workflow to me merged into main.
138148

139-
Regarding secrets, the forks are not allowed to use secrets by default.
140-
Every generated `GITHUB_TOKEN` has only read access to the repo/org without access to the secrets.
149+
Regarding secrets, the forks are not allowed to use secrets by default.
150+
Every generated `GITHUB_TOKEN` has only read access to the repo/org without access to the secrets.
151+
152+
## Testing workflows and actions
153+
Unit and integration tests invoked via [actions-tests.yml](../workflows/actions-tests.yml) workflow.
154+
It uses files specified within [tests](../tests) folder and via [act](https://github.com/nektos/act) it tries to execute the actions and check the outputs.
155+
Currently, we tests `check-permissions`, `generate-matrix`, and `parse-comment` actions.

‎.github/tests/events/parse-comment/issue_comment_11.json‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
{
2+
"event_name": "issue_comment",
3+
"action": "created",
4+
"comment": {
5+
"body": "/gha run tests=random-tests"
6+
},
7+
"issue": {
8+
"number": 42,
9+
"pull_request": {
10+
"head": {
11+
"ref": "feature-123",
12+
"sha": "1111111111111111111111111111111111111111",
13+
"repo": {
14+
"name": "test-repo",
15+
"owner": { "login": "batman" }
16+
}
17+
}
18+
}
19+
},
20+
"repository": {
21+
"name": "test-repo",
22+
"owner": { "login": "batman" }
23+
}
24+
}

‎.github/tests/events/parse-comment/issue_comment_closed_issue.json‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"event_name": "issue_comment",
3+
"action": "created",
4+
"comment": {
5+
"body": "/gha run profile=operators,operands"
6+
},
7+
"issue": {
8+
"number": 123,
9+
"state": "closed"
10+
},
11+
"repository": {
12+
"name": "test-repo",
13+
"owner": { "login": "batman" }
14+
}
15+
}

‎.github/tests/events/parse-comment/issue_comment_closed_pr.json‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
{
2+
"event_name": "issue_comment",
3+
"action": "created",
4+
"comment": {
5+
"body": "/gha run pipeline=regression,upgrade"
6+
},
7+
"issue": {
8+
"number": 42,
9+
"state": "closed",
10+
"pull_request": {
11+
"head": {
12+
"ref": "feature-123",
13+
"sha": "1111111111111111111111111111111111111111",
14+
"repo": {
15+
"name": "test-repo",
16+
"owner": { "login": "batman" }
17+
}
18+
}
19+
}
20+
},
21+
"repository": {
22+
"name": "test-repo",
23+
"owner": { "login": "batman" }
24+
}
25+
}

‎.github/tests/events/parse-comment/issue_comment_closed_pr_random_comment.json‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
{
2+
"event_name": "issue_comment",
3+
"action": "created",
4+
"comment": {
5+
"body": "This is random comment"
6+
},
7+
"issue": {
8+
"number": 42,
9+
"state": "closed",
10+
"pull_request": {
11+
"head": {
12+
"ref": "feature-123",
13+
"sha": "1111111111111111111111111111111111111111",
14+
"repo": {
15+
"name": "test-repo",
16+
"owner": { "login": "batman" }
17+
}
18+
}
19+
}
20+
},
21+
"repository": {
22+
"name": "test-repo",
23+
"owner": { "login": "batman" }
24+
}
25+
}

‎.github/tests/scenarios/parse-comment.yaml‎

Lines changed: 71 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ scenarios:
3939
fixture: .github/tests/events/parse-comment/issue_comment_3.json
4040
expectations:
4141
pipelines: ""
42-
profiles: "operators,operands,brokers-and-security,azp_kraft_upgrade,azp_kafka_upgrade"
42+
profiles: ""
4343
tests: ""
4444
groups: "nodeport"
4545
kafka: "4.1.0"
@@ -90,7 +90,7 @@ scenarios:
9090
fixture: .github/tests/events/parse-comment/issue_comment_6.json
9191
expectations:
9292
pipelines: ""
93-
profiles: "operators,operands,brokers-and-security,azp_kraft_upgrade,azp_kafka_upgrade"
93+
profiles: ""
9494
tests: ""
9595
groups: ""
9696
kafka: "latest"
@@ -107,7 +107,7 @@ scenarios:
107107
fixture: .github/tests/events/parse-comment/issue_comment_7.json
108108
expectations:
109109
pipelines: ""
110-
profiles: "operators,operands,brokers-and-security,azp_kraft_upgrade,azp_kafka_upgrade"
110+
profiles: ""
111111
tests: ""
112112
groups: ""
113113
kafka: "latest"
@@ -169,6 +169,23 @@ scenarios:
169169
kube: "latest"
170170
release: "latest"
171171

172+
- id: comment-random-tests
173+
description: "Pass tests param without specifying profile or pipeline"
174+
event: issue_comment
175+
fixture: .github/tests/events/parse-comment/issue_comment_11.json
176+
expectations:
177+
pipelines: ""
178+
profiles: ""
179+
tests: "random-tests"
180+
groups: ""
181+
kafka: "latest"
182+
strimzi_feature_gates: ""
183+
strimzi_rbac_scope: ""
184+
cluster_operator_install_type: "yaml"
185+
should_run: "true"
186+
kube: "latest"
187+
release: "latest"
188+
172189
- id: dispatch-default
173190
description: "Workflow dispatch with default values"
174191
event: workflow_dispatch
@@ -233,4 +250,55 @@ scenarios:
233250
cluster_operator_install_type: "yaml"
234251
should_run: "false"
235252
kube: "latest"
253+
release: "latest"
254+
255+
- id: comment-closed-pr
256+
description: "Comment on closed PR should not run"
257+
event: issue_comment
258+
fixture: .github/tests/events/parse-comment/issue_comment_closed_pr.json
259+
expectations:
260+
pipelines: "regression,upgrade"
261+
profiles: ""
262+
tests: ""
263+
groups: ""
264+
kafka: "latest"
265+
strimzi_feature_gates: ""
266+
strimzi_rbac_scope: ""
267+
cluster_operator_install_type: "yaml"
268+
should_run: "false"
269+
kube: "latest"
270+
release: "latest"
271+
272+
- id: comment-closed-issue
273+
description: "Comment on closed issue should not run"
274+
event: issue_comment
275+
fixture: .github/tests/events/parse-comment/issue_comment_closed_issue.json
276+
expectations:
277+
pipelines: ""
278+
profiles: "operators,operands"
279+
tests: ""
280+
groups: ""
281+
kafka: "latest"
282+
strimzi_feature_gates: ""
283+
strimzi_rbac_scope: ""
284+
cluster_operator_install_type: "yaml"
285+
should_run: "false"
286+
kube: "latest"
287+
release: "latest"
288+
289+
- id: comment-closed-pr-random-comment
290+
description: "Comment on closed issue should not run"
291+
event: issue_comment
292+
fixture: .github/tests/events/parse-comment/issue_comment_closed_pr_random_comment.json
293+
expectations:
294+
pipelines: ""
295+
profiles: ""
296+
tests: ""
297+
groups: ""
298+
kafka: "latest"
299+
strimzi_feature_gates: ""
300+
strimzi_rbac_scope: ""
301+
cluster_operator_install_type: "yaml"
302+
should_run: "false"
303+
kube: "latest"
236304
release: "latest"

0 commit comments

Comments
 (0)