forked from hashicorp/vault
-
Notifications
You must be signed in to change notification settings - Fork 1
691 lines (671 loc) · 34.4 KB
/
Copy pathbuild.yml
File metadata and controls
691 lines (671 loc) · 34.4 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
name: build
# Some words of caution before modifying this workflow:
# This file and workflow have been carefully architected to meet the following requirements:
# * Builds and tests the correct artifacts in both CE and Ent while maintaining a merge-conflict-free
# build.yml between the two repos
# * Supports multiple Github event triggers
# * Is highly optimized for cost and speed
# * Supports a variety of complex use cases
# If you wish to modify this file/workflow, please consider:
# * That the workflow must work under when triggered by pull_request, push, schedule, and
# workflow_dispatch events.
# * Merge-conflict-free compatibility between CE and Ent. Any changes that you make here must work
# in both repository contexts.
# * There are many workflow flow control modifiers. Further details below.
# * The total number of workers and the runner size. Further details below.
# Further details:
# * The workflow is used by the CRT system for building, notarizing, signing, and releasing
# artifacts. Whatever we do in this workflow must support building all artifacts and uploading
# them to Github in order to fulfill the CRT requirement, while also maintaining a smaller
# default build matrix for the pull requests.
# * CRT is designed to trigger a workflow called build in a workflow file called build.yml. This
# file must build the correct artifacts in CE and Ent, depending on the repository context.
# We've gone to great lengths to architect this file and workflow so that we can build and test
# the correct artifacts in each context while maintaining a merge-conflict-free file between CE
# and Ent. Any changes that you make here must work in both repository contexts.
# * The workflow must support multiple event triggers, all of which have varying event payloads
# which must be considered. If you make changes you must ensure that the workflow still works
# under normal pull_request, push, schedule, and workflow_dispatch trigger events.
# * The workflow has been highly optimized for cost and speed. If possible, it's better to add a
# step to an existing job than create another job. Over a long time horizon a new job is often
# much more expensive than a single step in an existing job, they also take up a limited number
# of our available runners.
# * Flow control in the workflow is complex in order to support many various use cases, including:
# * Only building on tier 1 supported "core" artifacts by default.
# * Only building the UI if the Go application or UI has been modified.
# * Skipping builds entirely if the commit or PR only modifies changelog or website documentation.
# * The ability to check out the HEAD reference instead of a Github merge branch reference.
# * The ability to control building all of our tier 2 supported "extended" artifacts via a
# build/all label, even if the event trigger is pull_request or, more importantly, a push.
# It's important to note that we must maintain support for building all artifacts on push
# via a pull request, even though push events aren't directly tied to pull requests. Our
# label metadata helpers are designed to handle this complexity.
# * The ability to build all of our artifacts on a scheduled cadence to ensure we don't
# accidentally regress.
# * All of these considerations, and many others, have led to the modular design we have here.
# * If you're doing something in more than one place, try and use small composite actions
# whenever possible.
on:
workflow_dispatch:
pull_request:
types:
- opened
- ready_for_review
- reopened
- synchronize
push:
branches:
- main
- release/**
- ce/**
schedule:
- cron: '05 02 * * *' # * is a special character in YAML so you have to quote this string
concurrency:
group: ${{ github.head_ref || github.run_id }}-build
cancel-in-progress: true
jobs:
setup:
# Setup is our entrypoint into the entire workflow. Here we gather metadata and export useful
# outputs for further use as inputs or for flow control.
#
# Trigger the setup workflow if any of the following conditions are true:
# * The workflow was triggered by a push (merge) to the main or release branch.
# * The workflow was triggered by pull request and the pull request is not a draft.
# * The workflow was triggered by on schedule to test building all artifacts.
if: |
github.event_name == 'push' ||
github.event_name == 'schedule' ||
(github.event_name == 'pull_request' && github.event.pull_request.draft == false)
runs-on: ${{ github.repository == 'hashicorp/vault' && 'ubuntu-latest' || fromJSON('["self-hosted","ubuntu-latest-x64"]') }}
permissions: write-all # vault-auth
outputs:
build-date: ${{ steps.metadata.outputs.vault-build-date }}
changed-files: ${{ steps.changed-files.outputs.changed-files }}
checkout-ref: ${{ steps.checkout.outputs.ref }}
compute-build: ${{ steps.metadata.outputs.compute-build }}
compute-build-ui: ${{ steps.metadata.outputs.compute-build-ui }}
compute-small: ${{ steps.metadata.outputs.compute-small }}
is-draft: ${{ steps.metadata.outputs.is-draft }}
is-ent-branch: ${{ steps.metadata.outputs.is-ent-branch }}
is-ent-repo: ${{ steps.metadata.outputs.is-ent-repo }}
is-fork: ${{ steps.metadata.outputs.is-fork }}
labels: ${{ steps.metadata.outputs.labels }}
vault-binary-name: ${{ steps.metadata.outputs.vault-binary-name }}
vault-revision: ${{ steps.metadata.outputs.vault-revision }}
vault-version: ${{ steps.metadata.outputs.vault-version }}
vault-version-metadata: ${{ steps.metadata.outputs.vault-version-metadata }}
vault-version-package: ${{ steps.metadata.outputs.vault-version-package }}
workflow-trigger: ${{ steps.metadata.outputs.workflow-trigger }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# Make sure we check out correct ref based on PR labels and such
- uses: ./.github/actions/checkout
id: checkout
# Get the vault version metadata
- uses: hashicorp/actions-set-product-version@2ec1b51402b3070bccf7ca95306afbd039e574ff # v2.0.1
id: set-product-version
with:
checkout: false # don't override the reference we've checked out
# Gather additional metadata about our execution context
- uses: ./.github/actions/metadata
id: metadata
with:
vault-version: ${{ steps.set-product-version.outputs.product-version }}
# Get the elevated github token
- if: steps.metadata.outputs.is-ent-repo == 'true'
id: vault-auth
name: Vault Authenticate
run: vault-auth
- if: steps.metadata.outputs.is-ent-repo == 'true'
id: vault-secrets
name: Fetch Vault Secrets
uses: hashicorp/vault-action@4c06c5ccf5c0761b6029f56cfb1dcf5565918a3b # v3.4.0
with:
url: ${{ steps.vault-auth.outputs.addr }}
caCertificate: ${{ steps.vault-auth.outputs.ca_certificate }}
token: ${{ steps.vault-auth.outputs.token }}
secrets: |
kv/data/github/${{ github.repository }}/github-token token | ELEVATED_GITHUB_TOKEN;
# Determine the changed files
- uses: ./.github/actions/changed-files
id: changed-files
with:
github-token: ${{ steps.metadata.outputs.is-ent-repo != 'true' && secrets.ELEVATED_GITHUB_TOKEN || steps.vault-secrets.outputs.ELEVATED_GITHUB_TOKEN }}
# Ensure that we have not changed any enterprise files on pull requests against ce/* branches.
# We do this here because we have the information, there's absolutely no reason to go
# further until we've resolved the issue, and we want to fail a required workflow if this
# issue is present.
- if: |
steps.metadata.outputs.is-ent-repo == 'true' &&
steps.metadata.outputs.workflow-trigger == 'pull_request' &&
startsWith(github.event.pull_request.base.ref, 'ce/') &&
contains(fromJSON(steps.changed-files.outputs.changed-files).groups, 'enterprise')
name: Ensure that we have not changed any enterprise files on pull requests against ce/* branches.
id: ce-ent-changed-file-check
run: |
echo "The pull request has changed files that are in enterprise groups!"
echo "If you believe this to be in error you will want to update the changed files checks in tools/pipeline/internal/pkg/changed"
echo "on our enterprise branches and backport them to ce/* before continuing with this pull request."
echo "See the 'changed-files' step above for a list of changed files and their associated metadata groups."
exit 1
# Make sure all required Go modules are cached at this point. We don't want all of the Go
# tests and build jobs to download modules and race to upload them to the cache.
- uses: ./.github/actions/set-up-go
name: Ensure Go modules are cached
with:
github-token: ${{ steps.metadata.outputs.is-ent-repo != 'true' && secrets.ELEVATED_GITHUB_TOKEN || steps.vault-secrets.outputs.ELEVATED_GITHUB_TOKEN }}
# Don't download them on a cache hit during setup, just make sure they're cached before
# subsequent workflows are run.
no-restore: true
# Make sure all required vault tools are cached at this point. We don't want all of the Go
# tests and build jobs to build the tools and race to upload them to the cache.
- uses: ./.github/actions/install-tools
name: Ensure Vault tools are cached
with:
# Don't download them on a cache hit during setup, just make sure they're cached before
# subsequent workflows are run.
no-restore: true
ce-code-checks:
# Checks that we want to run only on changes to ce/* branches that ought to pass but aren't
# required for the "completed-successfully" job to pass. This allows us to bubble up this
# information without gating merges, which can be useful if you have many PRs simultaneously
# modifying go.mod before CE PRs with the equivalent changes have been merged.
name: Check ce/* Pull Requests
if: |
needs.setup.outputs.is-ent-repo == 'true' &&
needs.setup.outputs.workflow-trigger == 'pull_request' &&
github.event.pull_request.draft == false &&
startsWith(github.event.pull_request.base.ref, 'ce/')
runs-on:
- self-hosted
- ubuntu-latest-x64
permissions: write-all # vault-auth
needs:
- setup
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
# Get the elevated github token
- id: vault-auth
name: Vault Authenticate
run: vault-auth
- id: vault-secrets
name: Fetch Vault Secrets
uses: hashicorp/vault-action@4c06c5ccf5c0761b6029f56cfb1dcf5565918a3b # v3.4.0
with:
url: ${{ steps.vault-auth.outputs.addr }}
caCertificate: ${{ steps.vault-auth.outputs.ca_certificate }}
token: ${{ steps.vault-auth.outputs.token }}
secrets: |
kv/data/github/${{ github.repository }}/github-token token | ELEVATED_GITHUB_TOKEN;
- id: set-up-pipeline
uses: ./.github/actions/set-up-pipeline
with:
github-token: ${{ steps.vault-secrets.outputs.ELEVATED_GITHUB_TOKEN }}
# Ensure that our go.mod files have not drifted if we've changed the Go toolchain on a CE PR
- if: |
needs.setup.outputs.is-ent-repo == 'true' &&
needs.setup.outputs.workflow-trigger == 'pull_request' &&
startsWith(github.event.pull_request.base.ref, 'ce/') &&
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'gotoolchain')
name: Check go.mod diff on pull requests against ce/* branches that changed the go toolchain
id: go-mod-diff
env:
GITHUB_TOKEN: ${{ steps.vault-secrets.outputs.ELEVATED_GITHUB_TOKEN }}
# Pass these in as env variables.
# See: https://docs.github.com/en/actions/reference/security/secure-use#good-practices-for-mitigating-script-injection-attacks
BASE_REF: ${{ github.event.pull_request.base.ref }}
HEAD_REF: ${{ github.event.pull_request.head.ref }}
run: |
# Check the deltas between all relevant go.mod files between the PR HEAD ref and the
# Enterprise equivalent base ref. We disable 'strict-require' because enterprise and CE
# are allowed to have different modules. We still enforce that like modules between
# branches share the same versions. Similarly, 'strict-replace' is also disabled to allow
# both modules to have different 'replace' directives but still compare like 'replace'
# directives that are shared. This allows enterprise to use enterprise plugin 'replace'
# directives and not fail this check. We also have this run in a non-failing job to handle
# cases where multiple PRs are changing go.mod in enteprise an CE.
# Strip the ce/ prefix off of the PR base ref
ent_base_ref="${BASE_REF#ce/}"
if [[ "$ent_base_ref" != "main" ]]; then
# If it's not main add the +ent suffix
ent_base_ref="${ent_base_ref}+ent"
fi
echo "A branch: $ent_base_ref"
echo "B branch: $HEAD_REF"
pipeline github check go-mod-diff ${{ runner.debug && '--log debug' }} \
--a-repo vault-enterprise \
--a-branch "$ent_base_ref" \
--b-repo vault-enterprise \
--b-branch "$HEAD_REF" \
--strict-replace=false \
--strict-require=false \
-p go.mod \
-p api/go.mod \
-p api/auth/approle/go.mod \
-p api/auth/aws/go.mod \
-p api/auth/azure/go.mod \
-p api/auth/cert/go.mod \
-p api/auth/gcp/go.mod \
-p api/auth/kubernetes/go.mod \
-p api/auth/ldap/go.mod \
-p api/auth/userpass/go.mod \
-p tools/pipeline/go.mod \
-p sdk/go.mod \
--format markdown | tee -a "$GITHUB_STEP_SUMMARY"
ui:
# The Web UI workflow is a prerequisite workflow for building our artifacts. If the application
# or UI change we'll trigger this workflow but only build it if we don't already have the asset
# in our Github cache.
#
# Ensure the Web UI is built if any of the following conditions are true:
# * The workflow was triggered by a push (merge) to the main or release branch.
# * The workflow was triggered by on schedule to test building all artifacts.
# * The `build/all` tag is present on either a pull request or on the pull request that created
# a merge
# * The workflow was triggered by a pull request, the pull request is not a draft, and the UI
# or app changed.
# * Our pipeline or test scenarios changed.
if: |
needs.setup.outputs.workflow-trigger == 'push' ||
needs.setup.outputs.workflow-trigger == 'schedule' ||
contains(fromJSON(needs.setup.outputs.labels), 'build/all') ||
(
needs.setup.outputs.workflow-trigger == 'pull_request' &&
needs.setup.outputs.is-draft == 'false' &&
(
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'ui') ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'gotoolchain') ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'pipeline') ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'enos') ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'app')
)
)
needs: setup
runs-on: ${{ fromJSON(needs.setup.outputs.compute-build-ui) }}
outputs:
cache-key: ui-${{ steps.ui-hash.outputs.ui-hash }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ needs.setup.outputs.checkout-ref }}
- name: Get UI hash
id: ui-hash
run: echo "ui-hash=$(git ls-tree HEAD ui --object-only)" | tee -a "$GITHUB_OUTPUT"
- name: Set up UI asset cache
id: cache-ui-assets
uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
with:
enableCrossOsArchive: true
lookup-only: true
path: http/web_ui
# Only restore the UI asset cache if we haven't modified anything in the ui directory.
# Never do a partial restore of the web_ui if we don't get a cache hit.
key: ui-${{ steps.ui-hash.outputs.ui-hash }}
- if: steps.cache-ui-assets.outputs.cache-hit != 'true'
name: Install PNPM
uses: pnpm/action-setup@41ff72655975bd51cab0327fa583b6e92b6d3061 # v4.2.0
with:
run_install: false
package_json_file: './ui/package.json'
- if: steps.cache-ui-assets.outputs.cache-hit != 'true'
name: Set up node and pnpm
uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version-file: ui/package.json
cache: pnpm
cache-dependency-path: ui/pnpm-lock.yaml
- if: steps.cache-ui-assets.outputs.cache-hit != 'true'
name: Build UI
run: make ci-build-ui
# Artifacts is where we'll build the various Vault binaries and package them into their respective
# Zip bundles, RPM and Deb packages, and container images. After we've packaged them we upload
# them to the Github Actions artifacts storage and execute our Enos test scenarios. If the
# workflow is triggered by a push to main CRT will take these artifacts from Github and perform
# all of the necessary notarizing and signing before uploading them to Artifactory.
#
# # Trigger the setup workflow if any of the following conditions are true:
#
# * The workflow was triggered by on schedule to test building all artifacts.
# * The Go app was changed.
# * Our pipeline or test scenarios changed.
# * The build/all label is present on a pull request or push.
artifacts-ce:
if: |
needs.setup.outputs.is-ent-branch != 'true' && (
needs.setup.outputs.workflow-trigger == 'schedule' ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'app') ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'gotoolchain') ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'pipeline') ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'enos') ||
contains(fromJSON(needs.setup.outputs.labels), 'build/all')
)
needs:
- setup
- ui # Don't build and test artifacts unless the UI build was triggered.
uses: ./.github/workflows/build-artifacts-ce.yml
with:
build-all: ${{contains(fromJSON(needs.setup.outputs.labels), 'build/all') || needs.setup.outputs.workflow-trigger == 'schedule' || contains(fromJSON(needs.setup.outputs.changed-files).groups, 'gotoolchain')}}
build-date: ${{ needs.setup.outputs.build-date }}
checkout-ref: ${{ needs.setup.outputs.checkout-ref }}
compute-build: ${{ needs.setup.outputs.compute-build }}
compute-small: ${{ needs.setup.outputs.compute-small }}
vault-revision: ${{ needs.setup.outputs.vault-revision }}
vault-version: ${{ needs.setup.outputs.vault-version }}
vault-version-package: ${{ needs.setup.outputs.vault-version-package }}
web-ui-cache-key: ${{ needs.ui.outputs.cache-key }}
secrets: inherit
artifacts-ent:
if: |
needs.setup.outputs.is-ent-branch == 'true' && (
needs.setup.outputs.workflow-trigger == 'schedule' ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'app') ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'gotoolchain') ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'pipeline') ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'enos') ||
contains(fromJSON(needs.setup.outputs.labels), 'build/all')
)
needs:
- setup
- ui # Don't build and test artifacts unless the UI build was triggered.
uses: ./.github/workflows/build-artifacts-ent.yml
with:
build-all: ${{contains(fromJSON(needs.setup.outputs.labels), 'build/all') || needs.setup.outputs.workflow-trigger == 'schedule' || contains(fromJSON(needs.setup.outputs.changed-files).groups, 'gotoolchain')}}
build-date: ${{ needs.setup.outputs.build-date }}
checkout-ref: ${{ needs.setup.outputs.checkout-ref }}
compute-build: ${{ needs.setup.outputs.compute-build }}
compute-small: ${{ needs.setup.outputs.compute-small }}
vault-revision: ${{ needs.setup.outputs.vault-revision }}
vault-version: ${{ needs.setup.outputs.vault-version }}
vault-version-package: ${{ needs.setup.outputs.vault-version-package }}
web-ui-cache-key: ${{ needs.ui.outputs.cache-key }}
secrets: inherit
# Setup HCP input parameters with proper validation
hcp-setup:
runs-on: ubuntu-latest
outputs:
pull-request-number: ${{ steps.pr-logic.outputs.pull-request-number }}
branch-name: ${{ steps.branch-logic.outputs.branch-name }}
steps:
- id: pr-logic
name: Determine pull request number
run: |
echo "Analyzing PR context for event: ${{ github.event_name }}"
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
PR_NUM="${{ github.event.pull_request.number }}"
if [[ -n "$PR_NUM" && "$PR_NUM" != "null" ]]; then
echo "PR context detected, using number: $PR_NUM"
else
echo "PR context but no valid number found, using 0"
PR_NUM="0"
fi
else
PR_NUM="0"
echo "Non-PR context, using fallback value: $PR_NUM"
fi
echo "pull-request-number=$PR_NUM" >> "$GITHUB_OUTPUT"
- id: branch-logic
name: Determine branch name
run: |
echo "Analyzing branch context for event: ${{ github.event_name }}"
if [[ "${{ github.event_name }}" == "schedule" ]]; then
BRANCH="main"
echo "Schedule context, using branch: $BRANCH"
else
BRANCH=""
echo "Non-schedule context, using empty branch"
fi
echo "branch-name=$BRANCH" >> "$GITHUB_OUTPUT"
hcp-image:
# Logic: Run HCP image job if:
# if needs.setup.outputs.is-ent-branch != 'true' then
# false
# elseif needs.setup.outputs.workflow-trigger == 'schedule' then
# true
# elseif needs.setup.outputs.workflow-trigger == 'pull_request' and (
# contains(fromJSON(needs.setup.outputs.changed-files).groups, 'hcp') or
# contains(fromJSON(needs.setup.outputs.labels), 'hcp/build-image') or
# contains(fromJSON(needs.setup.outputs.labels), 'hcp/test')
# ) then
# true
# end
if: |
( needs.setup.outputs.is-ent-branch != 'true' ) && false ||
( needs.setup.outputs.workflow-trigger == 'schedule' ) && true ||
( needs.setup.outputs.workflow-trigger == 'pull_request' &&
( contains(fromJSON(needs.setup.outputs.changed-files).groups, 'hcp') ||
contains(fromJSON(needs.setup.outputs.changed-files).groups, 'hcp') ||
contains(fromJSON(needs.setup.outputs.labels), 'hcp/build-image') ||
contains(fromJSON(needs.setup.outputs.labels), 'hcp/test')
)
) && true
needs:
- setup
- artifacts-ent
- hcp-setup
uses: ./.github/workflows/build-hcp-image.yml
with:
pull-request: ${{ fromJSON(needs.hcp-setup.outputs.pull-request-number) }}
branch: ${{ needs.hcp-setup.outputs.branch-name }}
create-aws-image: true
create-azure-image: false
hcp-environment: int
test:
# Test all of the testable artifacts if our repo isn't a fork. We don't test when the PR is
# created from a fork because secrets are not passed in and they are required.
if: |
always() &&
needs.setup.outputs.is-fork == 'false' &&
( needs.artifacts-ce.result == 'success' || needs.artifacts-ent.result == 'success' )
name: test ${{ matrix.artifact }}
needs:
- setup
- ui
- artifacts-ce
- artifacts-ent
uses: ./.github/workflows/test-run-enos-scenario-matrix.yml
strategy:
fail-fast: false
matrix:
include: ${{ needs.setup.outputs.is-ent-branch == 'true' && fromJSON(needs.artifacts-ent.outputs.testable-packages) || fromJSON(needs.artifacts-ce.outputs.testable-packages) }}
with:
build-artifact-name: ${{ matrix.artifact }}
runs-on: ${{ github.repository == 'hashicorp/vault' && '"ubuntu-latest"' || '["self-hosted","ubuntu-latest-x64"]' }}
sample-max: 1
sample-name: ${{ matrix.sample }}
ssh-key-name: ${{ github.event.repository.name }}-ci-ssh-key
vault-edition: ${{ matrix.edition }}
vault-revision: ${{ needs.setup.outputs.vault-revision }}
vault-version: ${{ needs.setup.outputs.vault-version-metadata }}
secrets: inherit
test-containers:
# Test all of the testable containers if our repo isn't a fork. We don't test when the PR is
# created from a fork because secrets are not passed in and they are required (for now).
if: |
always() &&
needs.setup.outputs.is-fork == 'false' &&
( needs.artifacts-ce.result == 'success' || needs.artifacts-ent.result == 'success' )
name: test ${{ matrix.artifact }}
needs:
- setup
- ui
- artifacts-ce
- artifacts-ent
uses: ./.github/workflows/test-run-enos-scenario-containers.yml
strategy:
fail-fast: false
matrix:
include: ${{ needs.setup.outputs.is-ent-branch == 'true' && fromJSON(needs.artifacts-ent.outputs.testable-containers) || fromJSON(needs.artifacts-ce.outputs.testable-containers) }}
with:
build-artifact-name: ${{ matrix.artifact }}
sample-max: 1
sample-name: ${{ matrix.sample }}
vault-edition: ${{ matrix.edition }}
vault-revision: ${{ needs.setup.outputs.vault-revision }}
vault-version: ${{ needs.setup.outputs.vault-version-metadata }}
secrets: inherit
test-hcp-image:
# Test our custom HCP image if our image build was successful and we've
# been configured with the correct label.
if: |
needs.setup.outputs.is-ent-branch == 'true' && (
( needs.setup.outputs.workflow-trigger == 'pull_request' &&
needs.artifacts-ent.result == 'success' &&
needs.hcp-image.result == 'success' &&
( contains(fromJSON(needs.setup.outputs.changed-files).groups, 'hcp') || contains(fromJSON(needs.setup.outputs.labels), 'hcp/test') )
) || (
needs.setup.outputs.workflow-trigger == 'schedule' &&
needs.artifacts-ent.result == 'success' &&
needs.hcp-image.result == 'success'
)
)
needs:
- setup
- artifacts-ent
- hcp-image
uses: ./.github/workflows/test-run-enos-scenario-cloud.yml
with:
hcp-environment: int
product-version: ${{ fromJSON(needs.hcp-image.outputs.image).product_version }}
completed-successfully:
# build/completed-successfully is the only build workflow that must pass in order to merge
# a pull request. This workflow is used to determine the overall status of all the prior
# workflows and to notify various different channels of success or failure. As part of this
# workflow we create the necessary build metadata that is required for the CRT build system.
#
# Our logic here mirrors that of setup as it and this are the only two workflows that must
# be triggered together.
if: |
always() &&
(
github.event_name == 'push' ||
github.event_name == 'schedule' ||
(github.event_name == 'pull_request' && github.event.pull_request.draft == false)
)
runs-on: ${{ github.repository == 'hashicorp/vault' && 'ubuntu-latest' || fromJSON('["self-hosted","ubuntu-latest-x64"]') }}
permissions: write-all # Ensure we have id-token:write access for vault-auth.
needs:
- setup
- ui
- artifacts-ce
- artifacts-ent
- hcp-image
- test
- test-containers
- test-hcp-image
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- id: disallow-merge-on-ce
if: |
needs.setup.outputs.workflow-trigger == 'pull_request' &&
github.repository == 'hashicorp/vault' &&
! contains(fromJSON(needs.setup.outputs.labels), 'build/all') &&
( github.event.pull_request.base.ref == 'main' || startsWith(github.event.pull_request.base.ref, 'release/') )
run: |
echo '::error::Merging into main and release/* branches in hashicorp/vault is disallowed. Use the copy-external-contributor workflow to copy and merge the change via hashicorp/vault-enterprise'
{
echo "result=failed"
echo "results=failed"
} | tee -a "$GITHUB_OUTPUT"
exit 1
- id: status
name: Determine status
run: |
results=$(tr -d '\n' <<< '${{ toJSON(needs.*.result) }}')
if ! grep -q -v -E '(failure|cancelled)' <<< "$results"; then
result="failed"
else
result="success"
fi
{
echo "result=${result}"
echo "results=${results}"
} | tee -a "$GITHUB_OUTPUT"
- if: needs.setup.outputs.is-ent-repo == 'true'
id: vault-auth
name: Vault Authenticate
run: vault-auth
- if: needs.setup.outputs.is-ent-repo == 'true'
id: secrets
name: Fetch Vault Secrets
uses: hashicorp/vault-action@4c06c5ccf5c0761b6029f56cfb1dcf5565918a3b # v3.4.0
with:
url: ${{ steps.vault-auth.outputs.addr }}
caCertificate: ${{ steps.vault-auth.outputs.ca_certificate }}
token: ${{ steps.vault-auth.outputs.token }}
secrets: |
kv/data/github/${{ github.repository }}/slack feed-vault-ci-official-webhook-url | slack-webhook-url;
- id: slackbot-webhook-url
run:
echo "slackbot-webhook-url=${{ needs.setup.outputs.is-ent-repo != 'true' && secrets.FEED_VAULT_CI_OFFICIAL_WEBHOOK_URL || steps.secrets.outputs.slackbot-webhook-url }}" >> "$GITHUB_OUTPUT"
- if: |
needs.setup.outputs.workflow-trigger == 'pull_request' &&
github.event.pull_request.head.repo.full_name == github.event.pull_request.base.repo.full_name &&
(github.repository == 'hashicorp/vault' || github.repository == 'hashicorp/vault-enterprise')
name: Create or update a build status comment on the pull request
env:
ARTIFACTS: ${{ needs.setup.outputs.is-ent-branch == 'true' && needs.artifacts-ent.result || needs.artifacts-ce.result }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.event.repository.name }}
RUN_ID: ${{ github.run_id }}
TEST: ${{ needs.test.result }}
TEST_CONTAINERS: ${{ needs.test-containers.result }}
UI: ${{ needs.ui.result }}
run: ./.github/scripts/report-build-status.sh
- name: Notify build failures in Slack
if: |
always() &&
steps.status.outputs.result != 'success' &&
(github.ref_name == 'main' || startsWith(github.ref_name, 'release/'))
uses: slackapi/slack-github-action@91efab103c0de0a537f72a35f6b8cda0ee76bf0a # v2.1.1
with:
errors: true # exit with an error if the payload is invalid
retries: rapid # retry if we're being rated limited
webhook: ${{ steps.slackbot-webhook-url.outputs.slackbot-webhook-url }}
webhook-type: incoming-webhook
payload: |
text: ${{ needs.setup.outputs.is-ent-branch == 'true' && 'vault-enterprise' || 'vault' }} build failures on ${{ github.ref_name }}
blocks:
- type: header
text:
type: plain_text
text: ':flashing-light: ${{ needs.setup.outputs.is-ent-branch == 'true' && 'vault-enterprise' || 'vault' }} build failures on ${{ github.ref_name }} :flashing-light:'
emoji: true
- type: divider
- type: section
text:
type: mrkdwn
text: |
${{ needs.setup.result == 'failure' && ':x: Setup' || '' }}
${{ needs.ui.result == 'failure' && ':x: Build UI' || '' }}
${{ needs.test.result == 'failure' && ':x: Enos package test scenarios' || '' }}
${{ needs.test-containers.result == 'failure' && ':x: Enos container test scenarios' || '' }}
${{ (needs.artifacts-ent.result == 'failure' || needs.artifacts-ce.result == 'failure') && ':x: Build Vault Artifacts' || '' }}
accessory:
type: button
text:
type: plain_text
text: View Failing Workflow
emoji: true
url: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
- uses: hashicorp/actions-generate-metadata@f1d852525201cb7bbbf031dd2e985fb4c22307fc # v1.1.3
# create build metadata if we successfully created artifacts
if: needs.artifacts-ent.result == 'success' || needs.artifacts-ce.result == 'success'
id: generate-metadata-file
with:
version: ${{ needs.setup.outputs.vault-version-metadata }}
product: ${{ needs.setup.outputs.vault-binary-name }}
- uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7.0.0
if: steps.generate-metadata-file.outcome == 'success' # upload our metadata if we created it
with:
name: metadata.json
path: ${{ steps.generate-metadata-file.outputs.filepath }}
if-no-files-found: error
- if: needs.setup.outputs.is-ent-branch == 'true' && needs.artifacts-ent.result == 'success'
id: generate-pao-metadata
uses: hashicorp-forge/actions-pao-tool/collect@6997f7457c338e008506005cc370e7b02f7fb421 # v1.0.3
- if: always() && steps.status.outputs.result != 'success'
name: Check for failed status
run: |
echo "One or more required build workflows failed: ${{ steps.status.outputs.results }}"
exit 1