Fix static-analysis scan pipeline short-circuiting and add scanner output completeness assertion by Copilot · Pull Request #58717 · github/gh-aw

Copilot · 2026-09-05T04:29:45Z

The static-analysis compilation pipeline aborted prematurely when an early scanner (e.g., zizmor) returned a finding error, silently skipping 6 of 8 security scanners (poutine, runner-guard, syft, grype, yamllint, shellcheck). This caused security scan runs to complete successfully while masking findings from 75% of the configured toolchain.

Changes

Non-short-circuiting batch scanner pipeline
Refactored compileSpecificFiles and compileAllFilesInDirectory in pkg/cli/compile_pipeline.go to use runBatchExternalTools. Batch scanners now execute sequentially to completion regardless of individual tool findings or errors, accumulating failures to return only after post-processing and results output complete.
Guaranteed scanner invocation logging
Updated actionlint.go, syft.go, grype.go, and shellcheck.go to ensure every enabled tool emits an execution log line to stderr even when operating on zero lock files, container images, or script steps.
Output completeness assertion step
Added an Assert static analysis output completeness step to .github/workflows/static-analysis-report.md that checks compile-output.txt for output from all 8 tools and triggers a hard failure if any scanner produces zero output.

Implementation

// pkg/cli/compile_pipeline.go: execute all enabled scanners sequentially without early return
func runBatchExternalTools(
	ctx context.Context,
	config CompileConfig,
	opts batchToolsOptions,
	stats *CompilationStats,
	validationResults *[]ValidationResult,
) (strictGrantErr error, batchToolErr error) {
	if err := runBatchLinters(ctx, config, opts); err != nil && batchToolErr == nil {
		batchToolErr = err
	}
	if err := runBatchDirScanners(ctx, config, opts); err != nil && batchToolErr == nil {
		batchToolErr = err
	}
	sGrantErr, containerErr := runBatchContainerScanners(ctx, config, opts, stats, validationResults)
	if sGrantErr != nil && strictGrantErr == nil {
		strictGrantErr = sGrantErr
	}
	if containerErr != nil && batchToolErr == nil {
		batchToolErr = containerErr
	}
	if err := runBatchScriptLinters(ctx, config, opts); err != nil && batchToolErr == nil {
		batchToolErr = err
	}
	return strictGrantErr, batchToolErr
}

Fixes [uk-ai-resilience] Static-analysis scan pipeline instrumentation gap: 6/8 tools produced zero output (Tier C) #58562

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2026-09-05T05:02:31Z

@copilot resolve the merge conflicts on this branch.

Copilot

🟡 Changes recommended

Moderate issues leave scanner completeness detection, empty-input logging, and regression coverage unreliable.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Prevents scanner failures from short-circuiting later static-analysis tools and adds output-completeness monitoring.

Changes:

Runs batch scanners sequentially while preserving errors.
Adds execution logging for empty scan inputs.
Adds workflow completeness checks and regression coverage.

File summaries

File	Description	Review notes
`pkg/cli/syft.go`	Logs zero-image scans.	No unresolved comments.
`pkg/cli/shellcheck.go`	Refactors helpers and logs zero-step scans.	Nit (2 votes): A new comment names the wrong function and contains an empty priority list.
`pkg/cli/grype.go`	Logs zero-image scans.	No unresolved comments.
`pkg/cli/compile_pipeline.go`	Adds non-short-circuiting scanner orchestration.	Moderate (1 vote): Actionlint still emits no marker with zero lock files. Nit (1 vote): A helper comment names the wrong function.
`pkg/cli/compile_external_tools_test.go`	Adds orchestration regression coverage.	Moderate (2 votes): Empty inputs do not test continued execution after an early scanner error.
`pkg/cli/actionlint.go`	Adds actionlint execution logging.	No unresolved comments.
`.github/workflows/static-analysis-report.md`	Adds scanner-output completeness assertions.	Moderate (2 votes): The shellcheck substring can be satisfied by actionlint output; use scanner-specific markers and regenerate the lock file.
`.github/workflows/static-analysis-report.lock.yml`	Regenerates the compiled workflow.	Must be regenerated after correcting the completeness assertion.

Review details

Suppressed comments (1)

pkg/cli/compile_pipeline.go:499

This comment names runBatchExternalTools, but it is attached to runBatchLinters; the actual orchestration function is declared later at line 646. Document the helper that follows here so generated documentation and code navigation are accurate.

// runBatchExternalTools executes all enabled batch analysis tools sequentially without short-circuiting
// when individual tools report findings or errors.

Files reviewed: 8/8 changed files
Comments generated: 4
Review effort level: Balanced

Copilot · 2026-09-05T05:51:04Z

+
+      MISSING_TOOLS=0
+      for tool in zizmor poutine actionlint runner-guard syft grype yamllint shellcheck; do
+        if ! grep -qi "$tool" "$COMPILE_LOG"; then


Fixed in 32f97f7: replaced the bare grep -qi "$tool" check with a per-scanner marker map (e.g. Running actionlint (, Running shellcheck on) so actionlint's "with shellcheck/pyflakes" message can no longer satisfy the dedicated shellcheck check. Verified locally that a log containing actionlint's message but missing the shellcheck marker now fails the assertion.

Correction: the commit hash is ffd49a9, not 32f97f7.

Copilot · 2026-09-05T05:51:10Z

+	stats := &CompilationStats{}
+	var validationResults []ValidationResult
+
+	strictGrantErr, batchToolErr := runBatchExternalTools(ctx, config, opts, stats, &validationResults)


Fixed in 32f97f7: added package-level function-variable seams for all 9 batch scanner entry points and rewrote the test to make actionlint return an error, then assert all scanners still run in pipeline order and the first error is preserved. Confirmed this test fails if the pipeline is reverted to short-circuit on the first error.

Correction: the commit hash is ffd49a9, not 32f97f7.

Copilot · 2026-09-05T05:51:16Z

+func runBatchLinters(ctx context.Context, config CompileConfig, opts batchToolsOptions) error {
+	var firstErr error
+
+	if config.Actionlint && !config.NoEmit && len(opts.lockFilesForActionlint) > 0 {


Fixed in 32f97f7: removed the len(lockFiles) > 0 guards for all lock-file batch tools (actionlint, zizmor, syft, grype, grant, yamllint) and for the directory scanners (poutine, runner-guard), so they're always invoked when enabled. runBatchLockFileTool and RunShellcheckOnLockFilesAndResources now emit an explicit zero-input stderr marker before returning, so the completeness check can tell "ran with zero input" apart from "never invoked".

Correction: the commit hash is ffd49a9, not 32f97f7.

Copilot · 2026-09-05T05:51:24Z

+// extractRunStepsFromLockFile parses a compiled lock file and returns all
+// run: steps whose effective shell is lintable by shellcheck.
+//
+// The effective shell for a step is resolved in priority order:


Fixed in 32f97f7: rewrote the doc comment on extractRunStepsFromJob to describe its actual behavior (job-level step extraction with the resolved default shell), removing the mislabeled/duplicated reference to extractRunStepsFromLockFile.

Correction: the commit hash is ffd49a9, not 32f97f7.

…nce-static-analysis-fix # Conflicts: # .github/workflows/static-analysis-report.lock.yml Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-09-05T05:19:21Z

@copilot resolve the merge conflicts on this branch.

Resolved in 73bb262 — merged main, resolved the lock-file conflict in static-analysis-report.lock.yml by regenerating it with gh aw compile, and verified all 299 workflows compile and pass validation.

gh-aw-bot · 2026-09-05T05:26:23Z

pr-sous-chef

@copilot Please take the next forward-progress pass on this PR.

A Copilot review is still requesting follow-up on scanner-output completeness assertions and regression coverage for continued execution after an early scanner error.
Please address the outstanding review feedback on current HEAD, then run the pr-finisher skill and summarize exactly what changed.
If one or more review comments are already fixed on HEAD, say that explicitly and resolve the related thread(s).

Generated by PR Sous Chef: https://github.com/github/gh-aw/actions/runs/33946898762

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 26.1 AIC · ⌖ 8.68 AIC · ⊞ 9.2K · ◷
_{Comment /souschef to run again}

…zero-input logging, doc fix Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-09-05T05:50:04Z

pr-sous-chef
```...

Addressed all four outstanding review threads in 32f97f7:

static-analysis-report.md completeness check: replaced the bare grep -qi "$tool" substring match with a per-scanner marker map (e.g. Running actionlint (, Running shellcheck on) so actionlint's "with shellcheck/pyflakes" summary can no longer satisfy the dedicated shellcheck check. Verified locally that a log missing the shellcheck marker (but containing actionlint's message) now correctly fails the assertion.
compile_external_tools_test.go: rewrote the test to inject an error from the first scanner (actionlint) via new package-level function-variable seams, and assert all 9 scanners still run in pipeline order with the first error preserved. Confirmed this test fails if the pipeline is reverted to short-circuit on the first error.
compile_pipeline.go:503: removed the len(lockFiles) > 0 guards that skipped invoking a tool (and its stderr marker) entirely when zero lock files were produced. runBatchLockFileTool and RunShellcheckOnLockFilesAndResources now emit an explicit zero-input marker so the completeness check distinguishes "ran with zero input" from "never invoked".
shellcheck.go:146-149: fixed the doc comment on extractRunStepsFromJob (it was mislabeled as documenting extractRunStepsFromLockFile and left the priority list empty).

Full pkg/cli test suite passes and all 299 workflows recompile successfully.

Initial plan

bce261f

Copilot AI assigned Copilot and pelikhan Sep 5, 2026

Copilot started work on behalf of pelikhan September 5, 2026 04:29 View session

Copilot AI linked an issue Sep 5, 2026 that may be closed by this pull request

[uk-ai-resilience] Static-analysis scan pipeline instrumentation gap: 6/8 tools produced zero output (Tier C) #58562

Open

github-actions Bot mentioned this pull request Sep 5, 2026

[static-analysis] Report - 2026-09-05 #58730

Open

Fix static analysis pipeline execution gap and output assertions

e56b5e8

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix static-analysis scan pipeline instrumentation gap~~ Fix static-analysis scan pipeline short-circuiting and add scanner output completeness assertion Sep 5, 2026

Copilot finished work on behalf of pelikhan September 5, 2026 05:01

Copilot AI requested a review from pelikhan September 5, 2026 05:01

pelikhan marked this pull request as ready for review September 5, 2026 05:02

Copilot AI balanced review requested due to automatic review settings September 5, 2026 05:02

Copilot started reviewing on behalf of pelikhan September 5, 2026 05:02 View session

Copilot AI reviewed Sep 5, 2026

View reviewed changes

Copilot started work on behalf of pelikhan September 5, 2026 05:10 View session

Merge remote-tracking branch 'origin/main' into copilot/uk-ai-resilie…

73bb262

…nce-static-analysis-fix # Conflicts: # .github/workflows/static-analysis-report.lock.yml Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan September 5, 2026 05:19

github-actions Bot mentioned this pull request Sep 5, 2026

[pr-sous-chef] PR Sous Chef run report #58638

Open

Copilot started work on behalf of gh-aw-bot September 5, 2026 05:26 View session

Address review feedback: scanner-specific markers, injectable seams, …

ffd49a9

…zero-input logging, doc fix Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

github-actions Bot mentioned this pull request Sep 5, 2026

[pr-review] Daily PR Code Quality Review — 33947957003 #58744

Open

Navigation Menu

Uh oh!

Fix static-analysis scan pipeline short-circuiting and add scanner output completeness assertion - #58717

Fix static-analysis scan pipeline short-circuiting and add scanner output completeness assertion#58717
pelikhan with Copilot wants to merge 4 commits into
mainfrom
copilot/uk-ai-resilience-static-analysis-fix

Copilot AI commented Sep 5, 2026 •

edited

Loading

Uh oh!

pelikhan commented Sep 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Sep 5, 2026

Uh oh!

Copilot AI Sep 5, 2026

Uh oh!

Copilot AI Sep 5, 2026

Uh oh!

Copilot AI Sep 5, 2026

Uh oh!

Copilot AI Sep 5, 2026

Uh oh!

Copilot AI Sep 5, 2026

Uh oh!

Copilot AI Sep 5, 2026

Uh oh!

Copilot AI Sep 5, 2026

Uh oh!

Copilot AI commented Sep 5, 2026

Uh oh!

gh-aw-bot commented Sep 5, 2026

Uh oh!

Copilot AI commented Sep 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

Copilot AI commented Sep 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Implementation

Uh oh!

pelikhan commented Sep 5, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Copilot AI Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Sep 5, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Sep 5, 2026

Uh oh!

gh-aw-bot commented Sep 5, 2026

Uh oh!

Copilot AI commented Sep 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Sep 5, 2026 •

edited

Loading