Skip to content

ci: tailor Copilot security reviews - #1012

Merged
ankitgoswami merged 1 commit into
mainfrom
ankitg/copilot-security-review
Sep 3, 2026
Merged

ci: tailor Copilot security reviews#1012
ankitgoswami merged 1 commit into
mainfrom
ankitg/copilot-security-review

Conversation

@ankitgoswami

Copy link
Copy Markdown
Contributor

Reviewable diff: +61/-164 across 4 files (excludes generated, test, and story files).

Summary

Copilot reviews now use a dedicated Proto Fleet security, correctness, and reliability skill instead of broad repository-wide authoring guidance. The skill keeps findings scoped to changed behavior, adds explicit cross-component and abuse-path checks, and requires ASD-STE100 Simplified Technical English. This makes the review instructions fit the team's review-only Copilot usage.

How it works

GitHub Copilot loads .github/skills/code-review/SKILL.md when it reviews a pull request. The skill directs it to map changed trust boundaries and data flows, trace behavior across Proto Fleet components, apply both generalized security checks and repository invariants, and report only concrete findings on the smallest relevant changed hunk. The automation path filter and labeler now recognize the skills directory instead of the removed global instructions file.

flowchart LR
    A["Pull request diff"] --> B["Copilot code review"]
    C["code-review skill"] --> B
    D["Repository context"] --> B
    B --> E["Material inline findings"]
Loading

Areas of the code involved

Area / package / file What changed Why it matters for review
.github/skills/code-review/SKILL.md Adds the review method, security boundary, generalized checks, Proto Fleet invariants, and finding format Defines the behavior and quality bar for Copilot reviews
.github/copilot-instructions.md Removes broad instructions intended for general coding work Prevents competing guidance when Copilot is used only for reviews
.github/path-filters.yml Treats .github/skills/** as a global change Keeps CI path selection aware of review-instruction changes
.github/labeler.yml Labels .github/skills/** changes as automation Preserves automatic classification for review tooling changes

Key technical decisions & trade-offs

  • Use a task-specific code-review skill instead of always-on Copilot instructions, so the guidance applies to reviews without shaping unused authoring workflows.
  • State the important checks explicitly instead of relying on generic reviewer defaults, at the cost of a longer skill that must be maintained with repository conventions.
  • Require concrete failure or abuse paths and changed-line findings instead of broad hardening advice, which favors precision over finding volume.
  • Keep the GitHub review-effort level as a repository setting; this code change cannot switch that UI setting from Lite to Balanced.

Testing & validation

  • Parsed the skill front matter successfully.
  • Parsed .github/path-filters.yml and .github/labeler.yml successfully.
  • Ran git diff --check successfully.
  • Did not run application tests because this change affects only GitHub review instructions and automation path metadata.
  • A live Copilot review and the Balanced review-effort setting still require verification in GitHub after merge or branch selection.

Add a review-specific skill with Proto Fleet security, correctness, and reliability checks. Remove broad Copilot authoring instructions so reviews use one focused instruction source.
Copilot AI lite review requested due to automatic review settings September 3, 2026 18:26
@ankitgoswami
ankitgoswami requested a review from a team as a code owner September 3, 2026 18:26
@github-actions github-actions Bot added documentation Improvements or additions to documentation automation labels Sep 3, 2026
@github-actions github-actions Bot added the review-policy: needs-review Managed by the Review Policy workflow. label Sep 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes are consistent across the skill file and the CI path/label metadata, and there are no remaining references to the removed instructions file.

Pull request overview

This pull request updates GitHub Copilot code review automation in this repo. It replaces broad authoring guidance with a dedicated code-review skill and updates CI metadata to treat skill changes as automation-impacting changes.

Changes:

  • Add .github/skills/code-review/SKILL.md to define a dedicated security, correctness, and reliability review skill for Copilot.
  • Update .github/path-filters.yml and .github/labeler.yml to include .github/skills/** as an automation-relevant path.
  • Remove .github/copilot-instructions.md to avoid conflicting global guidance.
File summaries
File Description
.github/skills/code-review/SKILL.md Adds the new Copilot code-review skill definition and review method.
.github/path-filters.yml Treats .github/skills/** as a global change for workflow path selection.
.github/labeler.yml Labels .github/skills/** changes as automation.
.github/copilot-instructions.md Removes the prior broad Copilot instructions file.
Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4f889b051b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread .github/skills/code-review/SKILL.md
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown

🔐 Codex Security Review

Note: This is an automated security-focused code review generated by Codex.
It should be used as a supplementary check alongside human review.
False positives are possible - use your judgment.

Scope summary

  • Reviewed pull request diff only (7e900a1369b6f36c8e5847848608ca8c60289b96...4f889b051b35579ec0ce370e9149d13497f6fef7, exact PR three-dot diff)
  • Model: gpt-5.6-sol

💡 Click "edited" above to see previous reviews for this PR.


Review Summary

Overall Risk: MEDIUM

Findings

[MEDIUM] Copilot code review does not consume the replacement instruction source

  • Category: Reliability
  • Location: .github/skills/code-review/SKILL.md:3
  • Description: GitHub Copilot code review reads repository instructions from .github/copilot-instructions.md. It does not activate Agent Skills from .github/skills/. The description text cannot make the code-review service load this skill. This PR deletes the supported instruction file, so the new review policy is not applied.
  • Impact: Copilot reviews silently lose the project-specific checks for authentication, plugin boundaries, pool hijacking, prompt injection, and other critical risks. This defeats the stated purpose of the change and can reduce defect detection in later pull requests.
  • Recommendation: Keep the review policy in .github/copilot-instructions.md. Retain the skill only for Copilot surfaces that support Agent Skills. Confirm the integration with a test pull request before removing the existing instruction file.

Notes

The YAML files parse successfully. This PR does not change application runtime code.


Generated by Codex Security Review |
Triggered by: @ankitgoswami |
Review workflow run

@ankitgoswami

Copy link
Copy Markdown
Contributor Author

🤖

GitHub Copilot code review reads repository instructions from .github/copilot-instructions.md. It does not activate Agent Skills from .github/skills/.

Not addressing: GitHub's current documentation says that Copilot code review can automatically use relevant repository-level agent skills and specifically recommends a review-focused directory such as .github/skills/code-review. This PR uses that supported location and name: https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-cloud-agent/add-skills#how-copilot-uses-agent-skills

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

All reviewed automation and guidance changes have no unresolved issues.

Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@ankitgoswami
ankitgoswami enabled auto-merge (squash) September 3, 2026 18:36
@github-actions github-actions Bot added review-policy: human-approved Managed by the Review Policy workflow. and removed review-policy: needs-review Managed by the Review Policy workflow. labels Sep 3, 2026
@ankitgoswami
ankitgoswami merged commit 978353b into main Sep 3, 2026
86 checks passed
@ankitgoswami
ankitgoswami deleted the ankitg/copilot-security-review branch September 3, 2026 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automation documentation Improvements or additions to documentation review-policy: human-approved Managed by the Review Policy workflow.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants