diff --git a/README.md b/README.md index 50038678..d8543bd5 100644 --- a/README.md +++ b/README.md @@ -172,29 +172,37 @@ All supported CLI argument inputs are [listed below](#arguments) with accompanyi | CLI | `working-directory` | Specify the working directory of TF code, alias of `arg-chdir`.
Example: `path/to/directory` | | CLI | `command` | Command to run between: `plan` or `apply`.1
Example: `plan` | | CLI | `tool` | Provisioning tool to use between: `terraform` or `tofu`.
Default: `terraform` | +| CLI | `build-artifacts` | File/directory paths to upload with plan and restore during apply.2
Example: `build/bundle.zip` | | CLI | `plan-file` | Supply existing plan file path instead of the auto-generated one.
Example: `path/to/file.tfplan` | | CLI | `pr-number` | Specify PR number in case of unsupported workflow trigger.
Example: `123` | | Check | `format` | Check format of TF code.
Default: `false` | | Check | `validate` | Check validation of TF code.
Default: `false` | -| Check | `plan-parity` | Replace plan file if it matches a newly-generated one to prevent stale apply.2
Default: `false` | -| Security | `plan-encrypt` | Encrypt plan file artifact with the given input.3
Example: `${{ secrets.PASSPHRASE }}` | +| Check | `plan-parity` | Replace plan file if it matches a newly-generated one to prevent stale apply.3
Default: `false` | +| Security | `plan-encrypt` | Encrypt plan file artifact with the given input.4
Example: `${{ secrets.PASSPHRASE }}` | | Security | `preserve-plan` | Preserve plan file "tfplan" in the given working directory after workflow execution.
Default: `false` | | Security | `upload-plan` | Upload plan file as GitHub workflow artifact.
Default: `true` | | Security | `retention-days` | Duration after which plan file artifact will expire in days.
Example: `90` | | Security | `token` | Specify a GitHub token.
Default: `${{ github.token }}` | | UI | `expand-diff` | Expand the collapsible diff section.
Default: `false` | | UI | `expand-summary` | Expand the collapsible summary section.
Default: `false` | -| UI | `comment-pr` | Add a PR comment: `always`, `on-diff`, or `never`.4
Default: `always` | -| UI | `comment-method` | PR comment by: `update` existing comment or `recreate` and delete previous one.5
Default: `update` | +| UI | `comment-pr` | Add a PR comment: `always`, `on-diff`, or `never`.5
Default: `always` | +| UI | `comment-method` | PR comment by: `update` existing comment or `recreate` and delete previous one.6
Default: `update` | | UI | `comment-pos-N` | Markdown content to render at various positions in the PR comment.
Example: `> [!NOTE]\n> Reviewed by security.` | -| UI | `tag-actor` | Tag the workflow triggering actor: `always`, `on-diff`, or `never`.4
Default: `always` | -| UI | `hide-args` | Hide comma-separated list of CLI arguments from the command input.6
Default: `detailed-exitcode,parallelism,lock,out,var=` | -| UI | `show-args` | Show comma-separated list of CLI arguments in the command input.6
Default: `workspace` | +| UI | `tag-actor` | Tag the workflow triggering actor: `always`, `on-diff`, or `never`.5
Default: `always` | +| UI | `hide-args` | Hide comma-separated list of CLI arguments from the command input.7
Default: `detailed-exitcode,parallelism,lock,out,var=` | +| UI | `show-args` | Show comma-separated list of CLI arguments in the command input.7
Default: `workspace` |
1. Both `command: plan` and `command: apply` include: `init`, `fmt` (with `format: true`), `validate` (with `validate: true`), and `workspace` (with `arg-workspace`) commands rolled into it automatically.
To separately run checks and/or generate outputs only, `command: init` can be used.

+1. The `build-artifacts` input accepts newline or comma-separated paths to files or directories that should be uploaded alongside the plan file and restored during apply. This is useful when using `data.archive_file` or other resources that generate interim build files needed during apply (e.g., zip archives for Lambda deployments).
+ Example usage: + ```yaml + build-artifacts: | + build/bundle.zip + dist/ + ``` 1. Originally intended for `merge_group` event trigger, `plan-parity: true` input helps to prevent stale apply within a series of workflow runs when merging multiple PRs.

1. The secret string input for `plan-encrypt` can be of any length, as long as it's consistent between encryption (plan) and decryption (apply).

1. The `on-diff` option is true when the exit code of the last TF command is non-zero (ensure `terraform_wrapper`/`tofu_wrapper` is set to `false`).

@@ -316,7 +324,6 @@ View [all notable changes](https://github.com/op5dev/tf-via-pr/releases "Release - Handling of inputs which contain space(s) (e.g., `working-directory: path to/directory`). - Handling of comma-separated inputs which contain comma(s) (e.g., `arg-var: token=1,2,3`); workaround with `TF_CLI_ARGS` [environment variable](https://developer.hashicorp.com/terraform/cli/config/environment-variables#tf_cli_args-and-tf_cli_args_name). -- Handling of interim build artifact(s) between `plan` and `apply` commands (e.g., zip archive); workaround with `arg-auto-approve: true` so that `apply` rebuilds artifact(s) for provisioning ([join discussion](https://github.com/op5dev/tf-via-pr/issues/517)).
diff --git a/action.yml b/action.yml index aa51bfc5..15f3669d 100644 --- a/action.yml +++ b/action.yml @@ -216,7 +216,8 @@ runs: gh api /repos/$GH_REPO/actions/artifacts/${artifact_id}/zip --header "$GH_API" --method GET > "$GH_IDENTIFIER_NAME.zip" # Unzip the plan file to the working directory, then clean up the zip file. - unzip "$GH_IDENTIFIER_NAME.zip" -d "$INPUTS_ARG_CHDIR" + unzip "$GH_IDENTIFIER_NAME.zip" -d "${INPUTS_ARG_CHDIR:-.}" + rm --force "${{ steps.identifier.outputs.name }}.zip" rm --force "$GH_IDENTIFIER_NAME.zip" - if: ${{ inputs.plan-encrypt != '' && steps.download.outcome == 'success' }} @@ -231,6 +232,64 @@ runs: openssl enc -aes-256-ctr -pbkdf2 -salt -in "$path.encrypted" -out "$path.decrypted" -pass file:"$temp_file" -d mv --force --verbose "$path.decrypted" "$path" + - if: ${{ steps.download.outcome == 'success' }} + id: restore-build-artifacts + env: + INPUTS_ARG_CHDIR: ${{ inputs.arg-chdir || inputs.working-directory }} + shell: bash + run: | + # Restore build artifacts from staging directory. + workdir="${INPUTS_ARG_CHDIR:-.}" + staging_dir="$workdir/.tfviapr-artifacts" + manifest_file="$staging_dir/manifest.json" + + # Skip if no build artifacts were staged. + if [[ ! -f "$manifest_file" ]]; then + echo "No build artifacts manifest found, skipping restore." + exit 0 + fi + + echo "Restoring build artifacts from manifest: $manifest_file" + + # Parse manifest and restore each artifact. + artifacts=$(jq -c '.artifacts[]' "$manifest_file" 2>/dev/null || echo "") + if [[ -z "$artifacts" ]]; then + echo "No artifacts in manifest." + rm -rf "$staging_dir" + exit 0 + fi + + while IFS= read -r artifact; do + rel_path=$(echo "$artifact" | jq -r '.path') + staging_name=$(echo "$artifact" | jq -r '.staging') + artifact_type=$(echo "$artifact" | jq -r '.type') + + # Determine destination path. + if [[ "$rel_path" = /* ]]; then + # Absolute path. + dest_path="$rel_path" + else + # Relative path - restore relative to working directory. + dest_path="$workdir/$rel_path" + fi + + # Create parent directory if needed. + dest_dir=$(dirname "$dest_path") + mkdir -p "$dest_dir" + + # Restore file or directory. + if [[ "$artifact_type" == "directory" ]]; then + cp -r "$staging_dir/$staging_name" "$dest_path" + else + cp "$staging_dir/$staging_name" "$dest_path" + fi + echo "Restored build artifact: $staging_dir/$staging_name -> $dest_path" + done <<< "$artifacts" + + # Clean up staging directory. + rm -rf "$staging_dir" + echo "Build artifacts restored successfully." + - if: ${{ steps.plan.outcome == 'success' || steps.download.outcome == 'success' }} env: INPUTS_TOOL: ${{ inputs.tool }} @@ -265,12 +324,72 @@ runs: rm --force "$path" fi + - if: ${{ inputs.build-artifacts != '' && steps.plan.outcome == 'success' }} + id: stage-build-artifacts + env: + INPUTS_ARG_CHDIR: ${{ inputs.arg-chdir || inputs.working-directory }} + INPUTS_BUILD_ARTIFACTS: ${{ inputs.build-artifacts }} + shell: bash + run: | + # Stage build artifacts for upload. + # Create staging directory and manifest file. + workdir="${INPUTS_ARG_CHDIR:-.}" + staging_dir="$workdir/.tfviapr-artifacts" + manifest_file="$staging_dir/manifest.json" + mkdir -p "$staging_dir" + echo '{"artifacts":[' > "$manifest_file" + + # Parse build-artifacts input (newline or comma-separated). + artifacts=$(echo "$INPUTS_BUILD_ARTIFACTS" | tr ',' '\n' | sed 's/^[[:space:]]*//;s/[[:space:]]*$//' | grep -v '^$') + first=true + + while IFS= read -r artifact_path; do + # Resolve path relative to working directory. + if [[ "$artifact_path" = /* ]]; then + # Absolute path - use as-is. + src_path="$artifact_path" + rel_path="$artifact_path" + else + # Relative path - resolve from working directory. + src_path="$workdir/$artifact_path" + rel_path="$artifact_path" + fi + + if [[ ! -e "$src_path" ]]; then + echo "Warning: Build artifact not found: $src_path" + continue + fi + + # Generate a unique staging name to avoid conflicts. + staging_name=$(echo "$rel_path" | md5sum | cut -d' ' -f1) + + # Copy file or directory to staging area. + if [[ -d "$src_path" ]]; then + cp -r "$src_path" "$staging_dir/$staging_name" + artifact_type="directory" + else + cp "$src_path" "$staging_dir/$staging_name" + artifact_type="file" + fi + + # Add to manifest. + if [[ "$first" != "true" ]]; then echo ',' >> "$manifest_file"; fi + first=false + echo "{\"path\":\"$rel_path\",\"staging\":\"$staging_name\",\"type\":\"$artifact_type\"}" >> "$manifest_file" + echo "Staged build artifact: $src_path -> $staging_dir/$staging_name" + done <<< "$artifacts" + + echo ']}' >> "$manifest_file" + echo "Build artifacts manifest created: $manifest_file" + - if: ${{ inputs.command == 'plan' && inputs.upload-plan == 'true' && (github.server_url == 'https://github.com' || contains(github.server_url, '.ghe.com')) }} id: upload uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 with: name: ${{ steps.identifier.outputs.name }} - path: ${{ format('{0}{1}tfplan{2}', inputs.arg-chdir || inputs.working-directory, (inputs.arg-chdir || inputs.working-directory) && '/' || '', inputs.plan-encrypt != '' && '.encrypted' || '') }} + path: | + ${{ format('{0}{1}tfplan{2}', inputs.arg-chdir || inputs.working-directory, (inputs.arg-chdir || inputs.working-directory) && '/' || '', inputs.plan-encrypt != '' && '.encrypted' || '') }} + ${{ inputs.build-artifacts != '' && format('{0}{1}.tfviapr-artifacts', inputs.arg-chdir || inputs.working-directory, (inputs.arg-chdir || inputs.working-directory) && '/' || '') || '' }} retention-days: ${{ inputs.retention-days }} overwrite: true @@ -279,9 +398,22 @@ runs: uses: actions/upload-artifact@c6a366c94c3e0affe28c06c8df20a878f24da3cf # v3.2.2 with: name: ${{ steps.identifier.outputs.name }} - path: ${{ format('{0}{1}tfplan{2}', inputs.arg-chdir || inputs.working-directory, (inputs.arg-chdir || inputs.working-directory) && '/' || '', inputs.plan-encrypt != '' && '.encrypted' || '') }} + path: | + ${{ format('{0}{1}tfplan{2}', inputs.arg-chdir || inputs.working-directory, (inputs.arg-chdir || inputs.working-directory) && '/' || '', inputs.plan-encrypt != '' && '.encrypted' || '') }} + ${{ inputs.build-artifacts != '' && format('{0}{1}.tfviapr-artifacts', inputs.arg-chdir || inputs.working-directory, (inputs.arg-chdir || inputs.working-directory) && '/' || '') || '' }} retention-days: ${{ inputs.retention-days }} + - if: ${{ inputs.build-artifacts != '' && (steps.upload.outcome == 'success' || steps.upload-v3.outcome == 'success') }} + env: + INPUTS_ARG_CHDIR: ${{ inputs.arg-chdir || inputs.working-directory }} + shell: bash + run: | + # Clean up build artifacts staging directory after upload. + workdir="${INPUTS_ARG_CHDIR:-.}" + staging_dir="$workdir/.tfviapr-artifacts" + rm -rf "$staging_dir" + echo "Cleaned up build artifacts staging directory." + - if: ${{ inputs.plan-parity == 'true' && (steps.download.outcome == 'success' || inputs.plan-file != '') }} env: INPUTS_ARG_CHDIR: ${{ inputs.arg-chdir || inputs.working-directory }} @@ -685,6 +817,10 @@ inputs: default: "true" description: "Upload plan file as GitHub workflow artifact (e.g., `true`)." required: false + build-artifacts: + default: "" + description: "Newline or comma-separated paths to files/directories to upload with plan and restore during apply (e.g., `build/bundle.zip`)." + required: false validate: default: "false" description: "Check validation of TF code (e.g., `false`)." diff --git a/tests/fail_build_artifact_missing/.gitignore b/tests/fail_build_artifact_missing/.gitignore new file mode 100644 index 00000000..d647a4a8 --- /dev/null +++ b/tests/fail_build_artifact_missing/.gitignore @@ -0,0 +1,8 @@ +# Terraform +.terraform/ +.terraform.lock.hcl +*.tfplan +tfplan + +# Build artifacts (generated during plan) +build/ diff --git a/tests/fail_build_artifact_missing/main.tf b/tests/fail_build_artifact_missing/main.tf new file mode 100644 index 00000000..9037676d --- /dev/null +++ b/tests/fail_build_artifact_missing/main.tf @@ -0,0 +1,92 @@ +# Test case demonstrating build artifact issue (#517) +# This replicates the scenario where data.archive_file generates a zip +# during plan, but the zip is missing during apply because only the +# tfplan file is uploaded/downloaded between workflow runs. +# +# To test this scenario (simulating separate plan/apply jobs): +# 1. Run: terraform init +# 2. Run: terraform plan -out=tfplan +# (This creates build/bundle.zip via data.archive_file) +# 3. Simulate artifact transfer: rm -rf build/ +# 4. Run: terraform apply tfplan +# (This FAILS because bundle.zip no longer exists) +# +# The new `build-artifacts` input solves this by allowing users to +# specify files/directories to upload alongside the tfplan file. +# +# Example workflow usage with build-artifacts: +# - uses: op5dev/tf-via-pr@v13 +# with: +# command: plan +# build-artifacts: build/bundle.zip +# # Or for multiple files/directories: +# # build-artifacts: | +# # build/bundle.zip +# # dist/ + +terraform { + required_providers { + archive = { + source = "hashicorp/archive" + version = "~> 2.0" + } + null = { + source = "hashicorp/null" + version = "~> 3.0" + } + } +} + +# Archive inline content - creates zip during plan phase +data "archive_file" "bundle" { + type = "zip" + output_path = "${path.module}/build/bundle.zip" + + source { + content = "exports.handler = async (event) => { return { statusCode: 200 }; };" + filename = "index.js" + } + + source { + content = jsonencode({ name = "lambda-function", version = "1.0.0" }) + filename = "package.json" + } +} + +# Simulates deploying the archive (e.g., to S3, Lambda, etc.) +# This will fail during apply if the zip file doesn't exist +resource "null_resource" "deploy" { + triggers = { + archive_hash = data.archive_file.bundle.output_base64sha256 + } + + provisioner "local-exec" { + command = <<-EOT + if [ -f "${data.archive_file.bundle.output_path}" ]; then + echo "SUCCESS: Archive found at ${data.archive_file.bundle.output_path}" + echo "Archive size: $(wc -c < "${data.archive_file.bundle.output_path}") bytes" + echo "Archive hash: ${data.archive_file.bundle.output_base64sha256}" + else + echo "ERROR: Archive missing at ${data.archive_file.bundle.output_path}" + echo "" + echo "This failure occurs when:" + echo " 1. 'terraform plan' runs in one workflow job (creates the zip)" + echo " 2. Only the tfplan file is transferred to another job" + echo " 3. 'terraform apply tfplan' runs but the zip is missing" + echo "" + echo "Solution: Use 'build-artifacts' input to transfer the zip file" + exit 1 + fi + EOT + } +} + +output "archive_path" { + value = data.archive_file.bundle.output_path + description = "Path to the generated archive file" +} + +output "archive_size" { + value = data.archive_file.bundle.output_size + description = "Size of the archive in bytes" +} diff --git a/tests/tf.sh b/tests/tf.sh index 1f8563bd..f531f2ae 100755 --- a/tests/tf.sh +++ b/tests/tf.sh @@ -4,3 +4,15 @@ terraform -chdir=tests/pass_one init -no-color 2> >(tee pass_one.txt) > >(tee pa terraform -chdir=tests/fail_format_diff fmt -check=true -diff=true -no-color 2> >(tee fail_format_diff.txt) > >(tee fail_format_diff.txt) terraform -chdir=tests/fail_data_source_error init -no-color 2> >(tee fail_data_source_error.txt) > >(tee fail_data_source_error.txt) terraform -chdir=tests/fail_invalid_resource_type init -no-color 2> >(tee fail_invalid_resource_type.txt) > >(tee fail_invalid_resource_type.txt) + +# Test case for build artifact issue (#517) +# This demonstrates the scenario where interim build artifacts are missing during apply +echo "=== Testing build artifact scenario (issue #517) ===" +terraform -chdir=tests/fail_build_artifact_missing init -no-color +terraform -chdir=tests/fail_build_artifact_missing plan -out=tfplan -no-color +# Simulate artifact transfer (only tfplan, not the build directory) +rm -rf tests/fail_build_artifact_missing/build/ +# This apply should fail because bundle.zip is missing +terraform -chdir=tests/fail_build_artifact_missing apply tfplan -no-color 2> >(tee fail_build_artifact_missing.txt) > >(tee fail_build_artifact_missing.txt) || echo "Expected failure: build artifact missing" +# Cleanup +rm -f tests/fail_build_artifact_missing/tfplan