Add PoC of pipelines check skill

[JanKrivanek]

Context

PoC of skill streamlining part of the kitten duty

@MichalPavlik - lets iterate on this as needed ;-)

Sample

[Copilot]

Pull request overview

Adds a new “pipelines-health-check” skill under .github/skills/ to support kitten-duty style health monitoring by collecting Azure DevOps pipeline run health and VS insertion PR check status, emitting structured JSON, and documenting how to present/investigate results.

Changes:

• Add PowerShell script to query recent Azure DevOps pipeline runs and extract failed task details from build timelines.
• Add PowerShell script to query active VS PRs assigned to the MSBuild reviewer group and summarize PR check status.
• Add SKILL.md describing how to run scripts, render overview tables, and drive deeper investigations via subagents.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 8 comments.

File	Description
.github/skills/pipelines-health-check/check-vs-pr-status.ps1	New script to query active/completed PRs and summarize required check status into JSON.
.github/skills/pipelines-health-check/check-pipeline-health.ps1	New script to query pipeline runs and extract failure details into JSON.
.github/skills/pipelines-health-check/SKILL.md	New skill documentation defining usage flow, reporting tables, and investigation templates.

Comments suppressed due to low confidence (2)

.github/skills/pipelines-health-check/SKILL.md:146

• Spelling: “offedning”, “distile”, “conscise” in this section. Fixing these will make the subagent prompt easier to follow/search.

  - identify which component/task is failing and check recent commits to main to try to identify offedning one.
5. If infrastructure issues - try to distile exact reason for the issue, check if there are other failing pipelines with the same issue or any open bugs for the issue. Put together conscise overview of the issue, along with the links to the failure messages. Suggest whom to contact for the further investigation

.github/skills/pipelines-health-check/SKILL.md:208

• Spelling: “anlyse” appears twice in this bullet list. Please correct to “analyze”/“analyse” consistently.

2. Ensure to acquire the (binlog-failure-analysis skill)[https://github.com/ViktorHofer/dotnet-skills/blob/main/msbuild-skills/skills/binlog-failure-analysis/SKILL.md] together with the binlog-mcp (spawn via `dnx -y baronfel.binlog.mcp@0.0.13`)
3. Use the binlog analysis skill and mcp to anlyse the binlog(s) you found and analyse problems from those

[rainersigwald]

Feels like these deterministic reports should be emitted by the scripts.

[JanKrivanek]

It still needs to chew through the data and formulate for itself what's needed for next steps - so no huge concerns from tokens point of view (unless we want to execute scripts standalone - which wasn't the plan).
I'd prefer less code and more expressive formulations - should be easier to understand what's happening.

But if you have strong opinion on that (e.g. would prefer to execute those scripts standalone in some cases) - feel free to perform any changes to your liking.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (4)

.github/skills/pipelines-health-check/check-vs-pr-status.ps1:74

• Same external-command error-handling issue as Get-ActivePRs: suppressing stderr and unconditionally ConvertFrom-Json-ing the output makes failures hard to diagnose and can crash with a JSON parse error. Add explicit $LASTEXITCODE/empty-output checks and a clear failure path.

    $json = az repos pr list `
        --repository $RepositoryId `
        --status completed `
        --reviewer $ReviewerId `
        --target-branch main `
        --top $Top `
        --organization $Organization `
        --project $Project `
        -o json 2>$null
    return $json | ConvertFrom-Json
}

.github/skills/pipelines-health-check/check-vs-pr-status.ps1:83

• az rest failures aren’t handled here; with 2>$null, an auth/permission failure can lead to $json being empty and ConvertFrom-Json throwing, or $resp.value being $null. Consider wrapping this in try/catch and checking $LASTEXITCODE/output so callers get either an empty statuses array or a descriptive error (instead of a parse exception).

function Get-PRStatuses {
    param([int]$PullRequestId)
    $url = "$Organization/$Project/_apis/git/repositories/$RepositoryId/pullrequests/$PullRequestId/statuses" +
           "?api-version=7.1"
    $json = az rest --method get --url $url --resource $script:AzDoResource 2>$null
    $resp = $json | ConvertFrom-Json
    return $resp.value
}

.github/skills/pipelines-health-check/check-pipeline-health.ps1:47

• az pipelines show is an external command; if it fails (not logged in / missing extension / access denied), PowerShell won’t throw and stderr is being suppressed. In that case $info will be $null or ConvertFrom-Json will throw a misleading JSON parse error. Consider checking $LASTEXITCODE and that the output is non-empty before ConvertFrom-Json, and surface a clear error message (optionally include captured stderr).

    param([int]$PipelineId)
    $info = az pipelines show --id $PipelineId --organization $Organization --project $Project --query "{name:name}" -o json 2>$null | ConvertFrom-Json
    return $info.name

.github/skills/pipelines-health-check/check-pipeline-health.ps1:60

• az pipelines runs list failures won’t be caught here (external commands don’t honor $ErrorActionPreference), and 2>$null hides the root cause; this can lead to ConvertFrom-Json exceptions on empty output. Add explicit handling for non-zero $LASTEXITCODE / empty $runsJson and emit a helpful error (or return an error-shaped JSON object) so the skill doesn’t fail with an opaque parse message.

    $runsJson = az pipelines runs list `
        --pipeline-id $PipelineId `
        --organization $Organization `
        --project $Project `
        --branch $Branch `
        --top $Top `
        -o json 2>$null
    return $runsJson | ConvertFrom-Json
}

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.