Fail activation when required skills cannot install by Copilot · Pull Request #58626 · github/gh-aw

Copilot · 2026-09-04T22:33:00Z

Declared skills could fail to install without failing the workflow, allowing the agent to run without required policy or safety guidance. Skill failures must instead stop agent execution while remaining visible in the failure issue or comment.

Failure handling
- Replace the collector warning with core.setFailed(...) after publishing structured failure outputs.
- This fails activation and prevents the dependent agent job from running.
Failure reporting
- Allow the conclusion job to run when skill_install_failure_count != '0'.
- Reuse the existing specialized skill-installation failure category and message with per-skill error details.

core.setOutput("failure_count", String(failureCount));
core.setOutput("errors", formattedErrors);

if (failureCount > 0) {
  core.setFailed(`${failureCount} skill(s) failed to install`);
}

Coverage
- Verify successful installs remain non-failing.
- Verify failed installs preserve outputs, fail activation, and enable conclusion reporting.

Fixes Skill install failure is only a warning, so the job succeeds without the skill #58446

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot

🟡 Changes recommended

An empty failure-count output incorrectly triggers conclusion jobs for skipped or cancelled activations.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Makes required skill-installation failures block agent execution while preserving structured failure reporting for issue #58446.

Changes:

Fails activation when skill installation errors are collected.
Enables conclusion reporting for skill failures.
Adds Go and JavaScript regression coverage.

File summaries

File	Description
`actions/setup/js/collect_skill_install_failures.cjs`	Marks collected failures as fatal.
`actions/setup/js/collect_skill_install_failures.test.cjs`	Tests success and failure behavior.
`pkg/workflow/notify_comment_conclusion_helpers.go`	Adds skill failures to the conclusion condition.
`pkg/workflow/notify_comment_test.go`	Tests generated condition inclusion.
`.github/workflows/ponytail-reviewer.lock.yml`	Regenerates the conclusion condition.
`.github/workflows/mattpocock-skills-reviewer.lock.yml`	Regenerates the conclusion condition.
`.github/workflows/impeccable-skills-reviewer.lock.yml`	Regenerates the conclusion condition.
`.github/workflows/daily-documentation-diagram.lock.yml`	Regenerates the conclusion condition.
`.github/workflows/archivx-agentic-workflows-analyzer.lock.yml`	Regenerates the conclusion condition.

Review details

Files reviewed: 9/9 changed files
Comments generated: 1
Review effort level: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

+	if len(data.SkillReferences) > 0 || len(data.Skills) > 0 {
+		skillInstallFailed := BuildNotEquals(BuildPropertyAccess(fmt.Sprintf("needs.%s.outputs.skill_install_failure_count", constants.ActivationJobName)), BuildStringLiteral("0"))
+		activationGuardrailsFailed = BuildOr(activationGuardrailsFailed, skillInstallFailed)
+	}


github-actions · 2026-09-04T23:04:38Z

⚠️ Security scanning failed for Ponytail Reviewer. Review the logs for details.

Lean already. Ship.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

ab.chatgpt.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "ab.chatgpt.com"

See Network Configuration for more information.

Generated by Ponytail Reviewer for #58626

github-actions · 2026-09-04T23:04:50Z

🧠 Matt Pocock Skills Reviewer failed during the skills-based review.

Warning

Threat Detection Engine Failure — The analysis engine could not complete. This is a tooling failure, not a security finding.

What happened

The threat detection engine failed to produce results.

Review the workflow run logs for details.

🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer

github-actions · 2026-09-04T23:04:59Z

✅ Design Decision Gate 🏗️ completed the design decision gate check. See the comment below for the result and any generated ADR draft.

No ADR enforcement needed: PR does not have the implementation label and has <=100 new lines of code in business logic directories.

🏗️ ADR gate enforced by Design Decision Gate 🏗️

github-actions · 2026-09-04T23:05:04Z

✅ Test Quality Sentinel completed test quality analysis.

Test Quality Sentinel skipped because pre-fetch PR data was unavailable: unable to fetch test file diff

🧪 Test quality analysis by Test Quality Sentinel

github-actions · 2026-09-04T23:05:09Z

✅ PR Code Quality Reviewer completed the code quality review.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

🔎 Code quality review by PR Code Quality Reviewer

github-actions · 2026-09-04T23:09:07Z

Comment Memory

reviewed_at: 2026-09-04T23:06:46Z
review_event: REQUEST_CHANGES
top_themes:
  - conclusion job can misfire on skipped activation because empty skill_install_failure_count satisfies != '0'
files_reviewed:
  - .github/workflows/archivx-agentic-workflows-analyzer.lock.yml
  - .github/workflows/daily-documentation-diagram.lock.yml
  - .github/workflows/impeccable-skills-reviewer.lock.yml
  - .github/workflows/mattpocock-skills-reviewer.lock.yml
  - .github/workflows/ponytail-reviewer.lock.yml
  - actions/setup/js/collect_skill_install_failures.cjs
  - actions/setup/js/collect_skill_install_failures.test.cjs
  - pkg/workflow/notify_comment_conclusion_helpers.go
  - pkg/workflow/notify_comment_test.go
comment_count: 0

Note

This comment is managed by comment memory.

It stores persistent context for this thread in the code block at the top of this comment.
Edit only the text inside the backtick fences; workflow metadata and the footer are regenerated automatically.

Learn more about comment memory

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 17.5 AIC · ⌖ 7.32 AIC · ⊞ 23.5K · ◷
_{Comment /review to run again}

github-actions

Requesting changes

The activation-failure path is headed in the right direction, but there is still a blocking edge case in the conclusion-job condition that can fire on skipped activation and create noise on workflows that never ran.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

🔎 Code quality review by PR Code Quality Reviewer · copilot · gpt54 · 17.5 AIC · ⌖ 7.32 AIC · ⊞ 23.5K
_{Comment /review to run again}

gh-aw-bot · 2026-09-04T23:34:11Z

pr-sous-chef

@copilot Please take the next forward-progress pass on this PR.

There is still blocking review feedback on the conclusion-job condition: when activation is skipped or cancelled, an empty skill_install_failure_count still satisfies != "0" and can fire the conclusion job on no-op runs.
Please guard that condition so it only treats the count as a failure when the output is non-empty and non-zero, then run the pr-finisher skill before handing back to maintainers.
Failed checks to review: agent https://github.com/github/gh-aw/actions/runs/33927521662/job/101201080609 ; agent https://github.com/github/gh-aw/actions/runs/33927521651/job/101200977405

Generated by PR Sous Chef: https://github.com/github/gh-aw/actions/runs/33929489073

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 31.9 AIC · ⌖ 8.65 AIC · ⊞ 9.2K · ◷
_{Comment /souschef to run again}

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-09-04T23:48:32Z

pr-sous-chef
```...

Updated buildConclusionJobCondition in pkg/workflow/notify_comment_conclusion_helpers.go to require a non-empty skill failure count before checking for a non-zero value. The focused test now covers both guards. Fix committed as d0efffb.

gh-aw-bot · 2026-09-04T23:49:06Z

@copilot PR sous-chef triage: please address the blocking review feedback, refresh the branch if needed, and run the pr-finisher skill once updates are pushed. Review items currently blocking merge: #58626 (review) and #58626 (review)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

github.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "github.com"

See Network Configuration for more information.

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 24.2 AIC · ⌖ 8.7 AIC · ⊞ 8.9K · ◷
_{Comment /souschef to run again}

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot · 2026-09-04T23:55:10Z

@copilot PR sous-chef triage: please address the blocking review feedback, refresh the branch if needed, and run the pr-finisher skill once updates are pushed. Review items currently blocking merge:...

The blocking review feedback is addressed in d0efffb: conclusion reporting now requires skill_install_failure_count to be non-empty and non-zero, preventing skipped or cancelled activations from firing the conclusion job. Targeted workflow tests pass; the branch is refreshed with the fix.

github-actions · 2026-09-05T01:05:18Z

Generated by 🔧 PR Triage Agent · copilot · mai10 · 24 AIC · ⌖ 2.63 AIC · ⊞ 15.1K · ◷

done

gh-aw-bot · 2026-09-05T01:56:14Z

pr-sous-chef

@copilot Please take the next forward-progress pass on this PR.

The current head appears to include your conclusion-job guard fix, but blocking review state is still visible on the PR.
Please reconcile the current head against the remaining review items, refresh the branch if needed, and run the pr-finisher skill before handing back to maintainers.
If the only remaining blocker is stale review state or routine maintainer attention rather than code changes, say that explicitly.

Generated by PR Sous Chef: https://github.com/github/gh-aw/actions/runs/33937269999

Generated by 👨‍🍳 PR Sous Chef · pi · gpt54 · 26.3 AIC · ⌖ 8.67 AIC · ⊞ 9.2K · ◷
_{Comment /souschef to run again}

Copilot · 2026-09-05T02:06:38Z

pr-sous-chef
```...

Reconciled the current head (4c06e16) against the remaining review feedback. The requested guard is present in d0efffb, targeted tests and final local formatting/lint/unit checks pass, and no further code changes are needed. The only remaining blocker is the outdated unresolved review state plus routine maintainer review; the review thread is stale and requires maintainer resolution.

Initial plan

8c66e6a

Copilot AI assigned Copilot and pelikhan Sep 4, 2026

Copilot started work on behalf of pelikhan September 4, 2026 22:33 View session

Copilot AI linked an issue Sep 4, 2026 that may be closed by this pull request

Skill install failure is only a warning, so the job succeeds without the skill #58446

Closed

Fail activation when required skills cannot install

0e29e8f

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~[WIP] Fix skill install failure warning without job failure~~ Fail activation when required skills cannot install Sep 4, 2026

Copilot finished work on behalf of pelikhan September 4, 2026 22:54

Copilot AI requested a review from pelikhan September 4, 2026 22:54

pelikhan marked this pull request as ready for review September 4, 2026 22:55

Copilot AI balanced review requested due to automatic review settings September 4, 2026 22:55

Copilot started reviewing on behalf of pelikhan September 4, 2026 22:56 View session

Copilot AI reviewed Sep 4, 2026

View reviewed changes

github-actions Bot previously requested changes Sep 4, 2026

View reviewed changes

github-actions Bot mentioned this pull request Sep 4, 2026

[aw] Matt Pocock Skills Reviewer timed out #58629

Open

github-actions Bot added the needs-review label Sep 4, 2026

github-actions Bot mentioned this pull request Sep 4, 2026

[aw] Impeccable Skills Reviewer timed out #58630

Open

github-actions Bot mentioned this pull request Sep 4, 2026

[pr-sous-chef] pr-sous-chef run report #58334

Closed

Copilot started work on behalf of gh-aw-bot September 4, 2026 23:34 View session

Copilot AI and others added 2 commits September 4, 2026 23:38

Plan conclusion failure guard

83ead63

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Guard empty skill failure counts

d0efffb

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot September 4, 2026 23:49

Copilot AI requested a review from gh-aw-bot September 4, 2026 23:49

Copilot started work on behalf of gh-aw-bot September 4, 2026 23:50 View session

Review PR sous-chef feedback

4c06e16

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot finished work on behalf of gh-aw-bot September 4, 2026 23:55

github-actions Bot mentioned this pull request Sep 5, 2026

[pr-sous-chef] PR Sous Chef run report #58638

Open

github-actions Bot added pr-action:fast_track pr-agent:copilot pr-batch:workflow-safety pr-priority:high pr-risk:high pr-type:bug labels Sep 5, 2026

github-actions Bot mentioned this pull request Sep 5, 2026

[PR Triage Report] PR triage report 2026-09-05 #58681

Closed

Copilot started work on behalf of gh-aw-bot September 5, 2026 01:56 View session

Copilot finished work on behalf of gh-aw-bot September 5, 2026 02:07

pelikhan merged commit 021f2ee into main Sep 5, 2026
43 checks passed

pelikhan deleted the copilot/fix-skill-install-warning-issue branch September 5, 2026 02:13

Uh oh!

Conversation

Copilot AI commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

github-actions Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 4, 2026

Comment Memory

Uh oh!

github-actions Bot left a comment

Choose a reason for hiding this comment

Requesting changes

Uh oh!

gh-aw-bot commented Sep 4, 2026

Uh oh!

Copilot AI commented Sep 4, 2026

Uh oh!

gh-aw-bot commented Sep 4, 2026

Uh oh!

Copilot AI commented Sep 4, 2026

Uh oh!

github-actions Bot commented Sep 5, 2026

Uh oh!

gh-aw-bot commented Sep 5, 2026

Uh oh!

Copilot AI commented Sep 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Sep 4, 2026 •

edited

Loading

github-actions Bot commented Sep 4, 2026 •

edited

Loading

github-actions Bot commented Sep 4, 2026 •

edited

Loading

github-actions Bot commented Sep 4, 2026 •

edited

Loading

github-actions Bot commented Sep 4, 2026 •

edited

Loading

github-actions Bot commented Sep 4, 2026 •

edited

Loading