Skip to content

Commit ee80cc7

Browse files
johnbierenjohnbieren
committed
refactor(RELEASE-2471): convert check-data-keys to python
This commit replaces the inline task script for the check-data-keys managed task with a standalone python script contained in the utils image. The tekton unit tests are updated accordingly. Assisted-By: Cursor Signed-off-by: Johnny Bieren <jbieren@redhat.com>
1 parent f728702 commit ee80cc7

23 files changed

Lines changed: 68 additions & 108 deletions

‎tasks/managed/check-data-keys/README.md‎

Lines changed: 18 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -8,23 +8,24 @@ meaning all the data keys must be allowed and formatted correctly.
88
For example, if `releaseNotes` is passed as a system and the data file does not have all the required
99
releaseNotes keys, the schema will give validation errors, and the task will fail.
1010

11-
The validation schema is defined in `schema/dataKeys.json` in this repository.
11+
The validation schema is bundled in the release-service-utils image at
12+
`/home/schemas/dataKeys.json`.
1213

1314
## Parameters
1415

15-
| Name | Description | Optional | Default value |
16-
|-------------------------|----------------------------------------------------------------------------------------------------------------------------|----------|------------------------------------------------------------------------------------------------------------------|
17-
| dataPath | Path to the JSON string of the merged data to use | No | - |
18-
| schema | URL to the JSON schema file to validate the data against | Yes | https://raw.githubusercontent.com/konflux-ci/release-service-catalog/refs/heads/development/schema/dataKeys.json |
19-
| systems | The systems to check that all data keys are present for | Yes | "" |
20-
| ociStorage | The OCI repository where the Trusted Artifacts are stored | Yes | empty |
21-
| ociArtifactExpiresAfter | Expiration date for the trusted artifacts created in the OCI repository. An empty string means the artifacts do not expire | Yes | 1d |
22-
| trustedArtifactsDebug | Flag to enable debug logging in trusted artifacts. Set to a non-empty string to enable | Yes | "" |
23-
| orasOptions | oras options to pass to Trusted Artifacts calls | Yes | "" |
24-
| sourceDataArtifact | Location of trusted artifacts to be used to populate data directory | Yes | "" |
25-
| dataDir | The location where data will be stored | Yes | /var/workdir/release |
26-
| taskGitUrl | The url to the git repo where the release-service-catalog tasks and stepactions to be used are stored | No | - |
27-
| taskGitRevision | The revision in the taskGitUrl repo to be used | No | - |
28-
| caTrustConfigMapName | The name of the ConfigMap to read CA bundle data from | Yes | trusted-ca |
29-
| caTrustConfigMapKey | The name of the key in the ConfigMap that contains the CA bundle data | Yes | ca-bundle.crt |
30-
| caCertPath | Path to CA certificate bundle for TLS verification with self-signed certificates | Yes | /mnt/trusted-ca/ca-bundle.crt |
16+
| Name | Description | Optional | Default value |
17+
|-------------------------|----------------------------------------------------------------------------------------------------------------------------|----------|-------------------------------|
18+
| dataPath | Path to the JSON string of the merged data to use | No | - |
19+
| schema | Path to the JSON schema file to validate the data against | Yes | /home/schemas/dataKeys.json |
20+
| systems | The systems to check that all data keys are present for | Yes | "" |
21+
| ociStorage | The OCI repository where the Trusted Artifacts are stored | Yes | empty |
22+
| ociArtifactExpiresAfter | Expiration date for the trusted artifacts created in the OCI repository. An empty string means the artifacts do not expire | Yes | 1d |
23+
| trustedArtifactsDebug | Flag to enable debug logging in trusted artifacts. Set to a non-empty string to enable | Yes | "" |
24+
| orasOptions | oras options to pass to Trusted Artifacts calls | Yes | "" |
25+
| sourceDataArtifact | Location of trusted artifacts to be used to populate data directory | Yes | "" |
26+
| dataDir | The location where data will be stored | Yes | /var/workdir/release |
27+
| taskGitUrl | The url to the git repo where the release-service-catalog tasks and stepactions to be used are stored | No | - |
28+
| taskGitRevision | The revision in the taskGitUrl repo to be used | No | - |
29+
| caTrustConfigMapName | The name of the ConfigMap to read CA bundle data from | Yes | trusted-ca |
30+
| caTrustConfigMapKey | The name of the key in the ConfigMap that contains the CA bundle data | Yes | ca-bundle.crt |
31+
| caCertPath | Path to CA certificate bundle for TLS verification with self-signed certificates | Yes | /mnt/trusted-ca/ca-bundle.crt |

‎tasks/managed/check-data-keys/check-data-keys.yaml‎

Lines changed: 13 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -16,15 +16,16 @@ spec:
1616
For example, if `releaseNotes` is passed as a system and the data file does not have all the required
1717
releaseNotes keys, the schema will give validation errors, and the task will fail.
1818
19-
The validation schema is defined in `schema/dataKeys.json` in this repository.
19+
The validation schema is bundled in the release-service-utils image at
20+
`/home/schemas/dataKeys.json`.
2021
params:
2122
- name: dataPath
2223
description: Path to the JSON string of the merged data to use
2324
type: string
2425
- name: schema
25-
description: URL to the JSON schema file to validate the data against
26+
description: Path to the JSON schema file to validate the data against
2627
type: string
27-
default: https://raw.githubusercontent.com/konflux-ci/release-service-catalog/refs/heads/development/schema/dataKeys.json
28+
default: /home/schemas/dataKeys.json
2829
- name: systems
2930
description: The systems to check that all data keys are present for
3031
type: string
@@ -127,42 +128,23 @@ spec:
127128
- name: caCertPath
128129
value: $(params.caCertPath)
129130
- name: check-data-keys
130-
image: quay.io/konflux-ci/release-service-utils@sha256:5546fa78d3c88d7b6a2e8cff8902f7757f00541d0bbaf113b9f293133894afa3
131+
image: quay.io/jbieren/release-service-utils:checkdata_1
131132
computeResources:
132133
limits:
133134
memory: 64Mi
134135
requests:
135136
memory: 64Mi # was exiting with code 137 when set to 32Mi
136137
cpu: 10m
137138
env:
138-
- name: "SCHEMA_FILE"
139+
- name: SCHEMA_FILE
139140
value: "$(params.schema)"
140-
script: |
141-
#!/usr/bin/env bash
142-
set -ex
143-
144-
if [ -f "/mnt/trusted-ca/ca-bundle.crt" ]; then
145-
export SSL_CERT_FILE="/mnt/trusted-ca/ca-bundle.crt"
146-
fi
147-
148-
if [ ! -f "$(params.dataDir)/$(params.dataPath)" ] ; then
149-
echo "No data JSON was provided."
150-
exit 1
151-
fi
152-
153-
schema="${SCHEMA_FILE/\.git\///}"
154-
if ! curl -sL --fail-with-body --retry 3 --retry-delay 5 --retry-all-errors "$schema" -o /tmp/schema ; then
155-
echo "Failed to download schema file: $schema"
156-
exit 1
157-
fi
158-
159-
# We want this to output the json without expansion
160-
# shellcheck disable=SC2016
161-
jq --argjson systems '$(params.systems)' '.systems += $systems' \
162-
"$(params.dataDir)/$(params.dataPath)" > "/tmp/systems"
163-
mv "/tmp/systems" "$(params.dataDir)/$(params.dataPath)"
164-
165-
check-jsonschema --output-format=text --schemafile "/tmp/schema" "$(params.dataDir)/$(params.dataPath)"
141+
- name: PARAM_DATA_DIR
142+
value: $(params.dataDir)
143+
- name: PARAM_DATA_PATH
144+
value: $(params.dataPath)
145+
- name: PARAM_SYSTEMS
146+
value: $(params.systems)
147+
command: ["/home/scripts/python/tasks/managed/check_data_keys.py"]
166148
- name: create-trusted-artifact
167149
computeResources:
168150
limits:

‎tasks/managed/check-data-keys/tests/mocks.sh‎

Lines changed: 0 additions & 15 deletions
This file was deleted.

‎tasks/managed/check-data-keys/tests/pre-apply-task-hook.sh‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,6 @@
11
#!/usr/bin/env bash
22

33
TASK_PATH="$1"
4-
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
5-
6-
# Inject mocks.sh into the task's first step
7-
yq -i '.spec.steps[1].script = load_str("'"$SCRIPT_DIR"'/mocks.sh") + .spec.steps[1].script' "$TASK_PATH"
84

95
# Add RBAC so that the SA executing the tests can retrieve configMap
106
kubectl apply -f .github/resources/crd_rbac.yaml
11-
12-
# Create a configMap with the schema to be used by the task
13-
kubectl delete configmap check-data-keys-schema --ignore-not-found
14-
kubectl create configmap check-data-keys-schema --from-file=dataKeys="$SCRIPT_DIR/../../../../schema/dataKeys.json"

‎tasks/managed/check-data-keys/tests/test-check-data-keys-cloud-marketplaces.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ spec:
4848
value: "$(params.trustedArtifactsDebug)"
4949
steps:
5050
- name: setup
51-
image: quay.io/konflux-ci/release-service-utils@sha256:5546fa78d3c88d7b6a2e8cff8902f7757f00541d0bbaf113b9f293133894afa3
51+
image: quay.io/jbieren/release-service-utils:checkdata_1
5252
script: |
5353
#!/usr/bin/env sh
5454
set -eux
@@ -128,7 +128,7 @@ spec:
128128
{"systemName": "releaseNotes", "dynamic": false}
129129
]
130130
- name: schema
131-
value: "https://get-local-schema.com"
131+
value: "/home/schemas/dataKeys.json"
132132
- name: ociStorage
133133
value: $(params.ociStorage)
134134
- name: orasOptions

‎tasks/managed/check-data-keys/tests/test-check-data-keys-dynamic-false-missing-data.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ spec:
5151
value: "$(params.trustedArtifactsDebug)"
5252
steps:
5353
- name: setup
54-
image: quay.io/konflux-ci/release-service-utils@sha256:5546fa78d3c88d7b6a2e8cff8902f7757f00541d0bbaf113b9f293133894afa3
54+
image: quay.io/jbieren/release-service-utils:checkdata_1
5555
script: |
5656
#!/usr/bin/env sh
5757
set -eux
@@ -86,7 +86,7 @@ spec:
8686
{"systemName": "releaseNotes", "dynamic": false}
8787
]
8888
- name: schema
89-
value: "https://get-local-schema.com"
89+
value: "/home/schemas/dataKeys.json"
9090
- name: ociStorage
9191
value: $(params.ociStorage)
9292
- name: orasOptions

‎tasks/managed/check-data-keys/tests/test-check-data-keys-dynamic-true-missing-data.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ spec:
4949
value: "$(params.trustedArtifactsDebug)"
5050
steps:
5151
- name: setup
52-
image: quay.io/konflux-ci/release-service-utils@sha256:5546fa78d3c88d7b6a2e8cff8902f7757f00541d0bbaf113b9f293133894afa3
52+
image: quay.io/jbieren/release-service-utils:checkdata_1
5353
script: |
5454
#!/usr/bin/env sh
5555
set -eux
@@ -84,7 +84,7 @@ spec:
8484
{"systemName": "releaseNotes", "dynamic": true}
8585
]
8686
- name: schema
87-
value: "https://get-local-schema.com"
87+
value: "/home/schemas/dataKeys.json"
8888
- name: ociStorage
8989
value: $(params.ociStorage)
9090
- name: orasOptions

‎tasks/managed/check-data-keys/tests/test-check-data-keys-fail-invalid-releasenotes-key.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ spec:
5252
value: "$(params.trustedArtifactsDebug)"
5353
steps:
5454
- name: setup
55-
image: quay.io/konflux-ci/release-service-utils@sha256:5546fa78d3c88d7b6a2e8cff8902f7757f00541d0bbaf113b9f293133894afa3
55+
image: quay.io/jbieren/release-service-utils:checkdata_1
5656
script: |
5757
#!/usr/bin/env sh
5858
set -eux
@@ -120,7 +120,7 @@ spec:
120120
{"systemName": "releaseNotes", "dynamic": false}
121121
]
122122
- name: schema
123-
value: "https://get-local-schema.com"
123+
value: "/home/schemas/dataKeys.json"
124124
- name: ociStorage
125125
value: $(params.ociStorage)
126126
- name: orasOptions

‎tasks/managed/check-data-keys/tests/test-check-data-keys-fail-malformed-cve-key.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ spec:
5151
value: "$(params.trustedArtifactsDebug)"
5252
steps:
5353
- name: setup
54-
image: quay.io/konflux-ci/release-service-utils@sha256:5546fa78d3c88d7b6a2e8cff8902f7757f00541d0bbaf113b9f293133894afa3
54+
image: quay.io/jbieren/release-service-utils:checkdata_1
5555
script: |
5656
#!/usr/bin/env sh
5757
set -eux
@@ -131,7 +131,7 @@ spec:
131131
{"systemName": "releaseNotes", "dynamic": false}
132132
]
133133
- name: schema
134-
value: "https://get-local-schema.com"
134+
value: "/home/schemas/dataKeys.json"
135135
- name: ociStorage
136136
value: $(params.ociStorage)
137137
- name: orasOptions

‎tasks/managed/check-data-keys/tests/test-check-data-keys-fail-missing-atlas.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ spec:
5858
value: "$(params.trustedArtifactsDebug)"
5959
steps:
6060
- name: setup
61-
image: quay.io/konflux-ci/release-service-utils@sha256:5546fa78d3c88d7b6a2e8cff8902f7757f00541d0bbaf113b9f293133894afa3
61+
image: quay.io/jbieren/release-service-utils:checkdata_1
6262
script: |
6363
#!/usr/bin/env sh
6464
set -eux
@@ -93,7 +93,7 @@ spec:
9393
{"systemName": "atlas", "dynamic": false}
9494
]
9595
- name: schema
96-
value: "https://get-local-schema.com"
96+
value: "/home/schemas/dataKeys.json"
9797
- name: ociStorage
9898
value: $(params.ociStorage)
9999
- name: orasOptions

0 commit comments

Comments
 (0)