-
Notifications
You must be signed in to change notification settings - Fork 133
Expand file tree
/
Copy pathapply-mapping.yaml
More file actions
788 lines (709 loc) · 37.3 KB
/
Copy pathapply-mapping.yaml
File metadata and controls
788 lines (709 loc) · 37.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
---
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: apply-mapping
annotations:
tekton.dev/pipelines.minVersion: "0.12.1"
tekton.dev/tags: release
spec:
description: |-
Tekton task to apply a mapping to a Snapshot.
The purpose of this task is to merge a mapping with the components contained in a Snapshot.
The mapping is expected to be present in the data field of the ReleasePlanAdmission provided in
the `releasePlanAdmissionPath`. If the data field does not contain a `mapping` key, the original
Snapshot is returned. If there is a `mapping` key, it is merged with the `components` key in the
Snapshot based on component name.
A `mapped` result is also returned from this task containing a simple true/false value that is
meant to inform whether a mapped Snapshot is being returned or the original one.
This task supports variable expansion in tag values from the mapping. The currently supported variables are:
* "{{ timestamp }}" -> The build-date label from the image in the format provided by timestampFormat or %s as the
default.
If the build-date label is not available, we use the Created field in the image metadata as a fallback.
* "{{ release_timestamp }}" -> The current time in the format provided by timestampFormat or %s as the default
* "{{ git_sha }}" -> The git sha that triggered the snapshot being processed
* "{{ git_short_sha }}" -> The git sha reduced to 7 characters
* "{{ digest_sha }}" -> The image digest of the respective component
* "{{ incrementer }}" -> Automatically finds the highest existing incremented tag in the
repository and generates the next sequential tag (e.g., if the highest tag is v1.0.0-2, it will generate v1.0.0-3)
* "{{ component-incrementer }}" -> Like {{ incrementer }}, but finds the highest existing tag
across ALL repositories in the component and generates the next sequential tag uniformly.
Use this instead of {{ incrementer }} when pushing to multiple registries to ensure every
registry receives the same tag (e.g., if repo-a has v1.0.0-3 and repo-b has v1.0.0-5,
both will receive v1.0.0-6).
* "{{ oci_version }}" -> The version from OCI image annotations (org.opencontainers.image.version), with fallback
to OCI image labels if not present in annotations (converts + to _ for tag compliance)
You can also expand image labels, e.g. "{{ labels.mylabel }}" -> The value of image label "mylabel"
params:
- name: snapshotPath
type: string
description: Path to the JSON string of the Snapshot spec in the config workspace to apply the mapping to
- name: dataPath
type: string
description: Path to the JSON string of the merged data to use in the data workspace
- name: failOnEmptyResult
type: string
description: Fail the task if the resulting snapshot contains 0 components
default: "false"
- name: ociStorage
description: The OCI repository where the Trusted Artifacts are stored
type: string
default: "empty"
- name: ociArtifactExpiresAfter
description: Expiration date for the trusted artifacts created in the
OCI repository. An empty string means the artifacts do not expire
type: string
default: "1d"
- name: trustedArtifactsDebug
description: Flag to enable debug logging in trusted artifacts. Set to a non-empty string to enable
type: string
default: ""
- name: orasOptions
description: oras options to pass to Trusted Artifacts calls
type: string
default: ""
- name: sourceDataArtifact
type: string
description: Location of trusted artifacts to be used to populate data directory
default: ""
- name: dataDir
description: The location where data will be stored
type: string
default: /var/workdir/release
- name: taskGitUrl
type: string
description: The url to the git repo where the release-service-catalog tasks and stepactions to be used are stored
- name: taskGitRevision
type: string
description: The revision in the taskGitUrl repo to be used
- name: caTrustConfigMapName
type: string
description: The name of the ConfigMap to read CA bundle data from
default: trusted-ca
- name: caTrustConfigMapKey
type: string
description: The name of the key in the ConfigMap that contains the CA bundle data
default: ca-bundle.crt
- name: addImplicitTimestampTag
type: string
description: |
When "true", for each repository the resolved {{ timestamp }} value is added to the list of
tags after translation (deduplicated). Fails if timestamp is empty. Only pipelines referencing
the check-labels task should set this to "true"
default: "false"
- name: caCertPath
type: string
description: Path to CA certificate bundle for TLS verification with self-signed certificates
default: /mnt/trusted-ca/ca-bundle.crt
results:
- name: mapped
type: string
description: A true/false value depicting whether or not the snapshot was mapped.
- name: sourceDataArtifact
type: string
description: Produced trusted data artifact
volumes:
- name: workdir
emptyDir: {}
- name: trusted-ca
configMap:
name: $(params.caTrustConfigMapName)
items:
- key: $(params.caTrustConfigMapKey)
path: ca-bundle.crt
optional: true
stepTemplate:
volumeMounts:
- mountPath: /var/workdir
name: workdir
- name: trusted-ca
mountPath: /mnt/trusted-ca
readOnly: true
env:
- name: IMAGE_EXPIRES_AFTER
value: $(params.ociArtifactExpiresAfter)
- name: "ORAS_OPTIONS"
value: "$(params.orasOptions)"
- name: "DEBUG"
value: "$(params.trustedArtifactsDebug)"
securityContext:
runAsUser: 1001
steps:
- name: use-trusted-artifact
computeResources:
limits:
memory: 64Mi
requests:
memory: 64Mi
cpu: 30m
ref:
resolver: "git"
params:
- name: url
value: $(params.taskGitUrl)
- name: revision
value: $(params.taskGitRevision)
- name: pathInRepo
value: stepactions/use-trusted-artifact/use-trusted-artifact.yaml
params:
- name: workDir
value: $(params.dataDir)
- name: sourceDataArtifact
value: $(params.sourceDataArtifact)
- name: caCertPath
value: $(params.caCertPath)
- name: apply-mapping
image: quay.io/konflux-ci/release-service-utils@sha256:5546fa78d3c88d7b6a2e8cff8902f7757f00541d0bbaf113b9f293133894afa3
computeResources:
limits:
memory: 64Mi
requests:
memory: 64Mi
cpu: "1"
script: |
#!/usr/bin/env bash
set -euxo pipefail
if [ -f "/mnt/trusted-ca/ca-bundle.crt" ]; then
export SSL_CERT_FILE="/mnt/trusted-ca/ca-bundle.crt"
fi
SNAPSHOT_SPEC_FILE="$(params.dataDir)/$(params.snapshotPath)"
DATA_FILE="$(params.dataDir)/$(params.dataPath)"
SNAPSHOT_SPEC_FILE_ORIG="${SNAPSHOT_SPEC_FILE}.orig"
if [ ! -f "${SNAPSHOT_SPEC_FILE}" ] ; then
echo "No valid snapshot file was found."
exit 1
fi
# Copy the original Snapshot spec file before overriding
cp "${SNAPSHOT_SPEC_FILE}" "${SNAPSHOT_SPEC_FILE_ORIG}"
if [ ! -f "${DATA_FILE}" ] ; then
echo "No data JSON file was found."
printf "false" | tee "$(results.mapped.path)"
exit 0
fi
MAPPING=$(jq '.mapping' "${DATA_FILE}")
if [[ $MAPPING == "null" ]] ; then
echo "Data file contains no mapping key."
printf "false" | tee "$(results.mapped.path)"
exit 0
fi
# Fetch and validate tags from a repository. Fails if the repo exists but
# returns an empty tag list (likely a transient registry error) to prevent
# silently falling back to increment 1 and overwriting existing tags.
extract_tags() {
local repo="$1"
local raw_json
raw_json=$(skopeo list-tags --retry-times 3 docker://"${repo}")
if ! jq -e '.Tags' <<< "${raw_json}" > /dev/null 2>&1; then
echo "Error: skopeo list-tags returned invalid response (missing Tags key) for ${repo}" >&2
exit 1
fi
local tag_count
tag_count=$(jq '.Tags | length' <<< "${raw_json}")
if [[ "${tag_count}" -eq 0 ]]; then
if skopeo inspect --retry-times 3 --no-tags --raw docker://"${repo}" > /dev/null 2>&1; then
echo "Error: Repository ${repo} exists but skopeo list-tags returned an empty tag list." >&2
echo "This may indicate a transient registry error. Failing to prevent overwriting" >&2
echo "existing tags by falling back to increment 1." >&2
exit 1
fi
echo ""
return
fi
jq -r '.Tags[]' <<< "${raw_json}"
}
# Function to handle incrementer logic
increment_tag() {
local tag_template="$1"
local repo="$2"
existing_tags=$(extract_tags "${repo}") || exit 1
# Remove `{{ incrementer }}` placeholder to get the version prefix for regex pattern
# shellcheck disable=SC2001
version_prefix=$(echo "${tag_template}" | sed 's/{{ incrementer }}//g')
# Match tags with 1–6 digit increments only. Ignore 7+ digit tags to avoid
# treating short commit SHAs as incrementer values
tag_pattern="^${version_prefix}[0-9]{1,6}$"
# Extract the numeric part of existing tags and find the max increment
max_increment=$(echo "${existing_tags}" | { grep -E "${tag_pattern}" || true; } \
| sed -E "s/${version_prefix}//" | sort -nr | head -n1)
# Calculate the next increment (default to 1 if max_increment is empty or unset)
# Use 10# to force decimal input preventing leading 0 from being treated as octal
increment=$((10#${max_increment:-0} + 1))
# Substitute `{{ incrementer }}` in the tag template with the calculated increment
tag="${tag_template//\{\{ incrementer \}\}/${increment}}"
# Validate the final tag format to avoid malformed tags
if [[ ! "${tag}" =~ ^[a-zA-Z0-9._-]+$ ]]; then
echo "Error: Invalid tag format after substitution: ${tag}"
exit 1
fi
echo "$tag" # Return the final tag
}
# Function to handle component-incrementer logic: finds the highest increment across
# ALL repositories in a component and returns the next uniform sequential tag.
# Results are cached by version_prefix to avoid redundant skopeo calls.
# Expected arguments are: [tag_template, all_repos_json]
component_increment_tag() {
local tag_template="$1"
local all_repos_json="${2:-[]}"
# Remove {{ component-incrementer }} placeholder to get the version prefix
local version_prefix
# shellcheck disable=SC2001
version_prefix=$(echo "${tag_template}" | sed 's/{{ *component-incrementer *}}//g')
# Return cached result if available for this prefix.
# Cache files survive subshell boundaries; associative arrays do not.
local cache_key
cache_key=$(printf '%s' "${version_prefix}" | base64 | tr -d '=\n' | tr '+/' '-_')
local cache_file="${_inc_cache_dir}/${cache_key}"
if [[ -f "${cache_file}" ]]; then
local cached_increment
cached_increment=$(< "${cache_file}")
local tag
# shellcheck disable=SC2001
tag=$(echo "${tag_template}" | sed "s/{{ *component-incrementer *}}/${cached_increment}/g")
if [[ ! "${tag}" =~ ^[a-zA-Z0-9._-]+$ ]]; then
echo "Error: Invalid tag format after substitution: ${tag}" >&2
exit 1
fi
echo "$tag"
return
fi
# Escape version_prefix for safe use in ERE (grep -E) and sed -E.
# Without escaping, a prefix like "v1.0.0-" would treat the dots as
# regex wildcards, potentially matching unintended tags.
local escaped_prefix
# shellcheck disable=SC2016
escaped_prefix=$(printf '%s' "${version_prefix}" | sed 's/[.[\\*^$()+?{|]/\\&/g')
# Match tags with 1–6 digit increments only. Ignore 7+ digit tags to avoid
# treating short commit SHAs as incrementer values
local tag_pattern="^${escaped_prefix}[0-9]{1,6}$"
# Find the global maximum increment across all repositories in the component
local global_max=0
local num_repos
num_repos=$(jq 'length' <<< "$all_repos_json")
for ((r = 0; r < num_repos; r++)); do
local repo
repo=$(jq -r --argjson r "$r" '.[$r]' <<< "$all_repos_json")
local existing_tags
existing_tags=$(extract_tags "${repo}") || exit 1
local repo_max
repo_max=$(echo "${existing_tags}" | { grep -E "${tag_pattern}" || true; } \
| sed -E "s/^${escaped_prefix}//" | sort -nr | head -n1)
# Use 10# to force decimal input preventing leading 0 from being treated as octal
if [[ -n "$repo_max" ]] && [[ $((10#${repo_max})) -gt $global_max ]]; then
global_max=$((10#${repo_max}))
fi
done
local increment=$((global_max + 1))
# Cache the result so subsequent repos in this component reuse the same value
echo "${increment}" > "${cache_file}"
local tag
# shellcheck disable=SC2001
tag=$(echo "${tag_template}" | sed "s/{{ *component-incrementer *}}/${increment}/g")
if [[ ! "${tag}" =~ ^[a-zA-Z0-9._-]+$ ]]; then
echo "Error: Invalid tag format after substitution: ${tag}" >&2
exit 1
fi
echo "$tag"
}
# Expected arguments are: [variable, substitute_map, labels_map]
substitute() {
variable=$1
substitute_map=$2
labels_map=$3
result=""
if [[ "$variable" == labels.* ]]; then
label="${variable#labels.}"
result="$(jq -r --arg labelval "$label" '.[$labelval] // ""' <<< "${labels_map}")"
else
result="$(jq -r --arg variable "$variable" '.[$variable] // ""' <<< "${substitute_map}")"
fi
echo "$result"
}
# When addImplicitTimestampTag is true, append the resolved timestamp value to the
# translated tag list (and deduplicate). Fails if timestamp is empty. Only used by
# the rh-advisories pipeline.
ensure_implicit_timestamp_value() {
local tags_json="$1"
local timestamp_val="$2"
if [ "$(params.addImplicitTimestampTag)" != "true" ]; then
echo "$tags_json"
return
fi
if [ -z "$timestamp_val" ]; then
echo "Error: addImplicitTimestampTag is true but timestamp is empty (no build-date or Created)." >&2
exit 1
fi
echo "$tags_json" | jq -c --arg ts "$timestamp_val" '. + [$ts] | unique'
}
# Expected arguments are [tags, substitute_map, labels_map, repo, all_repos_json]
# The tags argument is a json array
translate_tags () {
local tags=$1
local substitute_map=$2
local labels_map=$3
local repo=$4
local all_repos_json="${5:-[]}"
if [ "$tags" = '' ] ; then
echo ''
return
fi
local translated_tags='[]'
local NUM_TAGS
NUM_TAGS="$(jq 'length' <<< "${tags}")"
local i tag var_name replacement
for ((i = 0; i < NUM_TAGS; i++)); do
tag="$(jq -r --argjson i "$i" '.[$i]' <<< "${tags}")"
# Repeatedly translate {{}} references until none are left
while [[ $tag =~ \{\{\ *([[:alnum:]_\.-]+)\ *\}\} ]]; do
# Extract the variable name (e.g., timestamp), trimming any surrounding spaces
var_name="${BASH_REMATCH[1]}"
# Sanity check of the template variable name
if [[ ! "$var_name" =~ ^[a-zA-Z0-9._-]+$ ]]; then
echo "Error: Invalid variable name in tag definition: $var_name" >&2
exit 1
fi
# Handle incrementer logic
if [[ "$var_name" == "incrementer" ]]; then
tag=$(increment_tag "$tag" "$repo")
elif [[ "$var_name" == "component-incrementer" ]]; then
tag=$(component_increment_tag "$tag" "$all_repos_json")
else
replacement=$(substitute "$var_name" "$substitute_map" "$labels_map")
if [ -z "$replacement" ]; then
echo Error: Substitution variable unknown or empty: "$var_name" >&2
exit 1
fi
# Shellcheck suggests ${var//find/replace}, but
# that won't work here - we need to match arbitrary amount of spaces
# shellcheck disable=SC2001
tag="$(sed "s/{{ *$var_name *}}/$replacement/" <<< "$tag")"
fi
done
# Sanity check of the resulting tag value
if [[ ! "$tag" =~ ^[a-zA-Z0-9._-]+$ ]]; then
echo "Error: Invalid tag format: $tag" >&2
exit 1
fi
# Avoid duplicate tags - only add a tag if not already present
if [ "$(jq -c --arg tag "$tag" 'index($tag)' <<< "$translated_tags")" = null ]
then
translated_tags="$(jq -c --arg tag "$tag" '. + [$tag]' <<< "$translated_tags")"
fi
done
echo "$translated_tags"
}
convert_to_quay () { # Convert the registry.redhat.io URL to the quay.io format
local repository=$1
case "$repository" in
registry.redhat.io/*)
echo "${repository/registry.redhat.io/quay.io/redhat-prod}" \
| sed 's|/|----|g; s|quay.io----redhat-prod----|quay.io/redhat-prod/|'
;;
registry.stage.redhat.io/*)
echo "${repository/registry.stage.redhat.io/quay.io/redhat-pending}" \
| sed 's|/|----|g; s|quay.io----redhat-pending----|quay.io/redhat-pending/|'
;;
flatpaks.registry.redhat.io/*)
echo "${repository/flatpaks.registry.redhat.io/quay.io/rh-flatpaks-prod}" \
| sed 's|/|----|g; s|quay.io----rh-flatpaks-prod----|quay.io/rh-flatpaks-prod/|'
;;
flatpaks.registry.stage.redhat.io/*)
echo "${repository/flatpaks.registry.stage.redhat.io/quay.io/rh-flatpaks-stage}" \
| sed 's|/|----|g; s|quay.io----rh-flatpaks-stage----|quay.io/rh-flatpaks-stage/|'
;;
*)
echo "$repository"
;;
esac
}
# This block is temporary to support both quay.io and registry.redhat.io
# It should be removed once all repositories are migrated to registry.redhat.io
convert_to_registry () { # Convert the repository URL to the registry.redhat.io format
local repository=$1
case "$repository" in
quay.io/redhat-prod/*)
repository="${repository//quay.io\/redhat-prod/registry.redhat.io}"
repository="${repository//----//}"
echo "$repository"
;;
quay.io/redhat-pending/*)
repository="${repository//quay.io\/redhat-pending/registry.stage.redhat.io}"
repository="${repository//----//}"
echo "$repository"
;;
quay.io/rh-flatpaks-prod/*)
repository="${repository//quay.io\/rh-flatpaks-prod/flatpaks.registry.redhat.io}"
repository="${repository//----//}"
echo "$repository"
;;
quay.io/rh-flatpaks-stage/*)
repository="${repository//quay.io\/rh-flatpaks-stage/flatpaks.registry.stage.redhat.io}"
repository="${repository//----//}"
echo "$repository"
;;
registry.redhat.io/* | registry.stage.redhat.io/*)
# Return the original Red Hat registry paths
echo "$repository"
;;
*)
# Return empty for unhandled formats
echo ""
;;
esac
}
convert_to_registry_access () { # Convert the repository URL to the registry.access.redhat.com format
local repository=$1
case "$repository" in
registry.redhat.io/*)
echo "${repository/registry.redhat.io/registry.access.redhat.com}"
;;
registry.stage.redhat.io/*)
echo "${repository/registry.stage.redhat.io/registry.access.stage.redhat.com}"
;;
*)
echo ""
;;
esac
}
# Merge the mapping key contents in the data JSON file with the components key in the snapshot based
# on component name. Save the output as a compact JSON in the mapped_snapshot.json file in the workspace
{ echo -n "$(cat "${SNAPSHOT_SPEC_FILE_ORIG}")"; echo "${MAPPING}"; } | jq -c -s '
.[0] as $snapshot | .[0].components + .[1].components | group_by(.name) |
[.[] | select(length > 1)] | map(reduce .[] as $x ({}; . * $x)) as $mergedComponents |
$snapshot | .components = $mergedComponents' > "${SNAPSHOT_SPEC_FILE}"
printf "true" | tee "$(results.mapped.path)"
if [ "$(params.failOnEmptyResult)" = "true" ] && \
[ "$(jq '.components | length' < "${SNAPSHOT_SPEC_FILE}")" -eq 0 ]; then
echo "ERROR: Resulting snapshot contains 0 components. This means that there were 0 components present in"
echo "both your Snapshot and your ReleasePlanAdmission mapping. Take a look at your component names and"
echo "make sure that all components you want to release from the snapshot are present in the"
echo "ReleasePlanAdmission (by the name field of the component)."
echo "Components in snapshot: $(jq -c '[.components[].name]' "${SNAPSHOT_SPEC_FILE_ORIG}")"
echo "Components in mapping: $(jq -c '[.components[].name]' <<< "${MAPPING}")"
exit 1
fi
# Expand the tags in the data file
defaultTags=$(jq '.defaults.tags // []' <<< "$MAPPING")
defaultTimestampFormat=$(jq -r '.defaults.timestampFormat // "%s"' <<< "$MAPPING")
currentTimestamp="$(date "+%Y%m%d %T")"
defaultCGWSettings=$(jq -c '.defaults.contentGateway // {}' <<< "$MAPPING")
NUM_MAPPED_COMPONENTS=$(jq '.components | length' "${SNAPSHOT_SPEC_FILE}")
# File-based cache dir for component-incrementer results. A file-based
# approach is required because component_increment_tag is invoked inside
# $(...) subshells (via translate_tags), so bash associative array writes
# would be lost on subshell exit. Files persist across subshell boundaries.
_inc_cache_dir=$(mktemp -d)
trap 'rm -rf "${_inc_cache_dir}"' EXIT
for ((i = 0; i < NUM_MAPPED_COMPONENTS; i++)) ; do
# Clear the cache at the start of each component so that different
# components with the same tag template use independent repo sets.
rm -f "${_inc_cache_dir}/"* 2>/dev/null || true
component=$(jq -c --argjson i "$i" '.components[$i]' "${SNAPSHOT_SPEC_FILE}")
componentTags=$(jq '.componentTags // []' <<< "$component")
defaultComponentTags=$(jq -n --argjson defaults "$defaultTags" --argjson componentTags \
"$componentTags" '$defaults? + $componentTags? | unique')
# images are required to use sha reference - check this
NAME=$(jq -r '.name' <<< "$component")
IMAGE_REF=$(jq -r '.containerImage' <<< "$component")
if ! [[ "$IMAGE_REF" =~ ^.+@sha256:[0-9a-f]+$ ]] ; then
echo "Component ${NAME} contains an invalid containerImage value. sha reference is required: ${IMAGE_REF}"
exit 1
fi
git_sha=$(jq -r '.source.git.revision' <<< "$component") # this sets the value to "null" if it doesn't exist
build_sha=${IMAGE_REF##*:}
passedTimestampFormat=$(jq -r --arg default "$defaultTimestampFormat" \
'.timestampFormat // $default' <<< "$component")
release_timestamp="$(date -d "$currentTimestamp" "+$passedTimestampFormat")"
arch_json="$(get-image-architectures "${IMAGE_REF}")"
# The build-date label and Created values are not the same per architecture, but we don't support separate
# tags per arch. So, we just use the first digest listed.
arch="$(jq -rs 'map(.platform.architecture) | .[0]' <<< "$arch_json")"
os="$(jq -rs 'map(.platform.os) | .[0]' <<< "$arch_json")"
# Get first digest from architecture info to construct image reference
first_digest="$(jq -rs '.[0].digest' <<< "$arch_json")"
# Construct image reference with the first architecture's digest for annotations
image_with_digest="${IMAGE_REF%@*}@${first_digest}"
# Get raw manifest to extract annotations (works for all image types)
raw_manifest="$(skopeo inspect --retry-times 3 --no-tags --raw docker://"${image_with_digest}" | jq -c)"
annotations="$(jq -c '.annotations // {}' <<< "$raw_manifest")"
# Get config.mediaType from raw manifest to determine if this is a standard container image
config_media_type="$(jq -r '.config.mediaType // ""' <<< "$raw_manifest")"
# Get image metadata for labels, env, build_date
# Only standard container images support skopeo inspect without --raw
# Standard config types are:
# - application/vnd.oci.image.config.v1+json (OCI images)
# - application/vnd.docker.container.image.v1+json (Docker images)
# All other artifacts (Helm charts, ML models, empty configs, etc.) don't have
# labels/env and would fail with skopeo inspect
if [[ "$config_media_type" == "application/vnd.oci.image.config.v1+json" ]] || \
[[ "$config_media_type" == "application/vnd.docker.container.image.v1+json" ]]; then
# Standard container images - use standard skopeo inspect
image_metadata="$(skopeo inspect --retry-times 3 --no-tags \
--override-os "${os}" --override-arch "${arch}" docker://"${IMAGE_REF}" | jq -c)"
# For timestamp, use Labels.build-date and fallback to Created
build_date="$(jq -r '.Labels."build-date" // .Created // ""' <<< "$image_metadata")"
env_variables="$(jq -c '.Env // []' <<< "${image_metadata}")"
labels="$(jq -c '.Labels // {}' <<< "${image_metadata}")"
else
# Non-standard artifacts (Helm charts, ML models, etc.) don't support
# standard skopeo inspect - get build_date from annotations if available
build_date="$(jq -r '.["org.opencontainers.image.created"] // ""' <<< "$annotations")"
env_variables="[]"
labels="{}"
fi
# Get oci_version_raw from annotations, fallback to labels
oci_version_raw="$(jq -r '.["org.opencontainers.image.version"] // ""' <<< "$annotations")"
if [ -z "$oci_version_raw" ]; then
oci_version_raw="$(jq -r '.["org.opencontainers.image.version"] // ""' <<< "$labels")"
fi
# Add image env_variables metadata to component
if [ "$(jq 'length' <<< "$env_variables")" -ne 0 ] ; then
env_file=$(mktemp)
echo "$env_variables" > "$env_file"
jq --argjson i "$i" --slurpfile env "$env_file" \
'.components[$i].metadata = (.components[$i].metadata // {}) * {env_variables: $env[0]}' \
"${SNAPSHOT_SPEC_FILE}" > /tmp/temp && mv /tmp/temp "${SNAPSHOT_SPEC_FILE}"
fi
# Add image annotations metadata to component
if [ "$(jq 'length' <<< "$annotations")" -ne 0 ] ; then
annotations_file=$(mktemp)
# Convert annotations from {key: value} to [{name: key, value: value}]
jq -c 'if . then to_entries | map({name: .key, value: .value}) else [] end' \
<<< "$annotations" > "$annotations_file"
jq --argjson i "$i" --slurpfile annotations "$annotations_file" \
'.components[$i].metadata = (.components[$i].metadata // {}) * {annotations: $annotations[0]}' \
"${SNAPSHOT_SPEC_FILE}" > /tmp/temp && mv /tmp/temp "${SNAPSHOT_SPEC_FILE}"
fi
# Add image labels metadata to component
if [ "$(jq 'length' <<< "$labels")" -ne 0 ] ; then
labels_file=$(mktemp)
# Convert labels from {key: value} to [{name: key, value: value}]
jq -c 'if . then to_entries | map({name: .key, value: .value}) else [] end' \
<<< "$labels" > "$labels_file"
jq --argjson i "$i" --slurpfile labels "$labels_file" \
'.components[$i].metadata = (.components[$i].metadata // {}) * {labels: $labels[0]}' \
"${SNAPSHOT_SPEC_FILE}" > /tmp/temp && mv /tmp/temp "${SNAPSHOT_SPEC_FILE}"
fi
# Add media type to component
if [ -n "$config_media_type" ]; then
jq --argjson i "$i" --arg media_type "$config_media_type" \
'.components[$i].metadata = (.components[$i].metadata // {}) * {media_type: $media_type}' \
"${SNAPSHOT_SPEC_FILE}" > /tmp/temp && mv /tmp/temp "${SNAPSHOT_SPEC_FILE}"
fi
# Transform version to OCI tag format: replace + with _ (convention for OCI compliance)
# Set default value if empty (common for regular container images without OCI annotations)
oci_version="${oci_version_raw//+/_}"
oci_version="${oci_version:-unknown}"
if [ "${build_date}" = "" ] ; then
timestamp=""
else
timestamp="$(date -d "${build_date}" "+$passedTimestampFormat")"
fi
substitute_map="$(jq -n -c \
--arg timestamp "${timestamp}" \
--arg release_timestamp "${release_timestamp}" \
--arg git_sha "${git_sha}" \
--arg git_short_sha "${git_sha:0:7}" \
--arg digest_sha "${build_sha}" \
--arg oci_version "${oci_version}" \
'$ARGS.named')"
# Also substitute filename values in the staged section of components
STAGED_FILES=$(jq '.staged.files | length' <<< "$component")
for ((j = 0; j < STAGED_FILES; j++)) ; do
file=$(jq -c --argjson j "$j" '.staged.files[$j]' <<< "$component")
filenameArrayPreSubstitution=$(jq '.filename' <<< "$file" | jq -cs)
# {{ incrementer }} is not supported in staged.files values, so we just pass
# "" as the repo argument
subbedFilename=$(translate_tags "${filenameArrayPreSubstitution}" \
"${substitute_map}" "${labels}" ""| jq -r '.[0]')
jq --argjson i "$i" --argjson j "$j" --arg filename "$subbedFilename" \
'.components[$i].staged.files[$j].filename = $filename' "${SNAPSHOT_SPEC_FILE}" > /tmp/temp \
&& mv /tmp/temp "${SNAPSHOT_SPEC_FILE}"
done
# apply defaults for contentGateway
componentCGWSettings=$(jq -c '.contentGateway // {}' <<< "$component")
updatedComponentCGWSettings=$(merge-json "$defaultCGWSettings" "$componentCGWSettings")
componentCGWSettingsSize=$(jq '. | length' <<< "${updatedComponentCGWSettings}")
if [ "${componentCGWSettingsSize}" -gt "0" ]; then
jq --argjson i "$i" --argjson componentCGWSettings "$updatedComponentCGWSettings" \
'.components[$i].contentGateway = $componentCGWSettings' \
"${SNAPSHOT_SPEC_FILE}" > /tmp/temp && mv /tmp/temp "${SNAPSHOT_SPEC_FILE}"
fi
# Build a JSON array of all repository URLs for this component.
# This is used by {{ component-incrementer }} to query all repos and compute a
# uniform increment value across registries.
component_repos_json='[]'
_ci_num_repos=$(jq '.repositories | length' <<< "$component")
for ((_ci_j = 0; _ci_j < _ci_num_repos; _ci_j++)) ; do
_ci_repo_url=$(jq -r --argjson j "$_ci_j" '.repositories[$j].url' <<< "$component")
component_repos_json=$(jq -c --arg url "$_ci_repo_url" '. + [$url]' \
<<< "$component_repos_json")
done
NUM_REPOSITORIES=$(jq '.repositories | length' <<< "$component")
for ((j = 0; j < NUM_REPOSITORIES; j++)) ; do
repository=$(jq -c --argjson j "$j" '.repositories[$j]' <<< "$component")
repoTags=$(jq '.tags // []' <<< "$repository")
url=$(jq -r '.url' <<< "$repository")
echo "Processing component: $NAME, repository: $url"
allTagsPreSubstitution=$(jq -n --argjson defaults "$defaultComponentTags" --argjson repoTags \
"$repoTags" '$defaults? + $repoTags? | unique')
tags=$(translate_tags "${allTagsPreSubstitution}" "${substitute_map}" "${labels}" "${url}" \
"${component_repos_json}")
tags=$(ensure_implicit_timestamp_value "${tags}" "${timestamp}")
if [ "$(jq 'length' <<< "$tags")" -gt 0 ] ; then
jq --argjson i "$i" --argjson j "$j" --argjson updatedTags "$tags" \
'.components[$i].repositories[$j].tags = $updatedTags' "${SNAPSHOT_SPEC_FILE}" > /tmp/temp \
&& mv /tmp/temp "${SNAPSHOT_SPEC_FILE}"
fi
# This block is temporary to support both quay.io and registry.redhat.io
# It should be removed once all repositories are migrated to registry.redhat.io
if [[ "$url" == quay.io/redhat-prod/* ||
"$url" == quay.io/redhat-pending/* ||
"$url" == quay.io/rh-flatpaks-prod/* ||
"$url" == quay.io/rh-flatpaks-stage/* ]]; then
url=$(convert_to_registry "$url")
fi
# Convert to registry and quay format
if [[ "$url" == registry.redhat.io/* ||
"$url" == registry.stage.redhat.io/* ||
"$url" == flatpaks.registry.redhat.io/* ||
"$url" == flatpaks.registry.stage.redhat.io/* ]]; then
rh_registry_repo=$url
registry_access_repo=$(convert_to_registry_access "$url")
url=$(convert_to_quay "$url")
jq --argjson i "$i" \
--argjson j "$j" \
--arg url "$url" \
--arg rh_registry_repo "$rh_registry_repo" \
--arg registry_access_repo "$registry_access_repo" \
'.components[$i].repositories[$j].url = $url |
.components[$i].repositories[$j]["rh-registry-repo"] = $rh_registry_repo |
.components[$i].repositories[$j]["registry-access-repo"] = $registry_access_repo' \
"${SNAPSHOT_SPEC_FILE}" > /tmp/temp && mv /tmp/temp "${SNAPSHOT_SPEC_FILE}"
fi
done
done
- name: create-trusted-artifact
computeResources:
limits:
memory: 128Mi
requests:
memory: 128Mi
cpu: 250m
ref:
resolver: "git"
params:
- name: url
value: $(params.taskGitUrl)
- name: revision
value: $(params.taskGitRevision)
- name: pathInRepo
value: stepactions/create-trusted-artifact/create-trusted-artifact.yaml
params:
- name: ociStorage
value: $(params.ociStorage)
- name: workDir
value: $(params.dataDir)
- name: sourceDataArtifact
value: $(results.sourceDataArtifact.path)
- name: caCertPath
value: $(params.caCertPath)