Skip to content

Commit 72bf502

Browse files
committed
ci: add notify-on-failure job to catch silent CI skips
Opens/updates one 'ci-alert' issue (and auto-closes it when green) if a key job fails or update_nmisp_py is skipped — the silent-staleness mode behind #440. test_ipynb_colab is intentionally unwatched (chronically red 030 dead-kernel). Uses GITHUB_TOKEN + job-scoped issues: write, no new secrets.
1 parent 3e8d4d6 commit 72bf502

1 file changed

Lines changed: 77 additions & 0 deletions

File tree

.github/workflows/conda_env_test.yml

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -249,3 +249,80 @@ jobs:
249249
with:
250250
commit_message: "Update nmisp_py"
251251
repository: nmisp_py_dest
252+
253+
notify-on-failure:
254+
# Proactive alert so a silent regression can't go unnoticed — e.g. the #440
255+
# case where update_nmisp_py silently 'skipped' for a long time with no red
256+
# signal. Opens/updates one 'ci-alert' issue when a watched job FAILS, or
257+
# when update_nmisp_py is SKIPPED, and auto-closes it once the watched jobs
258+
# pass. test_ipynb_colab is intentionally NOT watched (chronically red 030
259+
# dead-kernel) to avoid alerting on every run.
260+
needs: [test_utils, badges, test_ipynb_linux, test_ipynb_native, update_nmisp_py]
261+
if: always() && github.event_name != 'fork' && github.actor != 'dependabot[bot]'
262+
name: Notify on CI failure
263+
runs-on: ubuntu-latest
264+
permissions:
265+
issues: write
266+
steps:
267+
- name: Alert on failed or skipped key jobs
268+
uses: actions/github-script@v7
269+
env:
270+
NEEDS: ${{ toJSON(needs) }}
271+
with:
272+
script: |
273+
const needs = JSON.parse(process.env.NEEDS);
274+
// Result values that count as a problem, per job. For update_nmisp_py,
275+
// 'skipped' is also bad — that is the silent-staleness signal (#440).
276+
const watch = {
277+
test_utils: ['failure', 'cancelled'],
278+
badges: ['failure', 'cancelled'],
279+
test_ipynb_linux: ['failure', 'cancelled'],
280+
test_ipynb_native: ['failure', 'cancelled'],
281+
update_nmisp_py: ['failure', 'cancelled', 'skipped'],
282+
};
283+
const bad = Object.keys(watch)
284+
.filter(j => needs[j] && watch[j].includes(needs[j].result))
285+
.map(j => `${j} → ${needs[j].result}`);
286+
287+
const { owner, repo } = context.repo;
288+
const LABEL = 'ci-alert';
289+
const runUrl = `${context.serverUrl}/${owner}/${repo}/actions/runs/${context.runId}`;
290+
const sha = context.sha.slice(0, 7);
291+
292+
// ensure the label exists (idempotent)
293+
try {
294+
await github.rest.issues.getLabel({ owner, repo, name: LABEL });
295+
} catch {
296+
await github.rest.issues.createLabel({ owner, repo, name: LABEL,
297+
color: 'd73a4a', description: 'Automated CI failure / silent-skip alert' }).catch(() => {});
298+
}
299+
300+
const open = (await github.rest.issues.listForRepo({
301+
owner, repo, state: 'open', labels: LABEL, per_page: 1 })).data[0];
302+
303+
if (bad.length === 0) {
304+
if (open) {
305+
await github.rest.issues.createComment({ owner, repo, issue_number: open.number,
306+
body: `✅ Watched jobs green again on \`${context.ref}\` (\`${sha}\`). Auto-closing.\n${runUrl}` });
307+
await github.rest.issues.update({ owner, repo, issue_number: open.number, state: 'closed' });
308+
}
309+
return;
310+
}
311+
312+
const body = [
313+
`🔴 **CI alert** on \`${context.ref}\` (commit \`${sha}\`)`,
314+
'',
315+
'Problem jobs:',
316+
...bad.map(b => `- \`${b}\``),
317+
'',
318+
`Run: ${runUrl}`,
319+
'',
320+
'_test_ipynb_colab is not watched (chronically red — 030 dead-kernel). This issue auto-closes when the watched jobs pass._',
321+
].join('\n');
322+
323+
if (open) {
324+
await github.rest.issues.createComment({ owner, repo, issue_number: open.number, body });
325+
} else {
326+
await github.rest.issues.create({ owner, repo,
327+
title: '🔴 CI alert: a key job failed or was skipped', labels: [LABEL], body });
328+
}

0 commit comments

Comments
 (0)