Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 20 additions & 10 deletions .github/workflows/ci-tests-custom-nodes-cloud.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,16 @@
name: 'CI: Tests Custom Nodes Cloud'

on:
pull_request:
branches-ignore: [wip/*, draft/*, temp/*]
# No pull_request trigger. Every automatic run shares ONE serial Cloud
# resource (the smoke account's queue), and GitHub keeps a single pending
# slot per concurrency group - so per-PR pushes stampeded the group and
# evicted each other instead of queueing (16 of the 40 runs before this
# change were cancelled; none of the PR-triggered survivors produced a
# verdict PRs acted on). PR-time custom-node coverage stays with the core
# gate and the pure specs in the main shards; Cloud verdicts come from
# suite-branch/main pushes, the merge queue, and manual dispatch.
push:
branches: [main, master]
branches: [main, master, nathaniel/custom-node-e2e-suite]
merge_group:
workflow_dispatch:

Expand All @@ -60,13 +66,14 @@ concurrency:
# newer one queues into the group - under 3+ concurrent triggers the middle
# run is cancelled, not queued. This is not safe for a required check. Do not
# mark this check required while pending runs can be evicted from this group.
# workflow_dispatch runs get their own group: main-push runs share the
# instance group and each new pending run EVICTS the older pending one
# (GitHub keeps one pending slot per group) - calibration dispatches
# 30316897509/30317780440/30318407927 died queued or displaced under main
# churn. Keep dispatches isolated from automatic runs and preserve serial
# execution while whole-queue observation remains part of the proof.
group: ${{ github.event_name == 'workflow_dispatch' && 'custom-nodes-cloud-dispatch' || 'custom-nodes-cloud-instance' }}
# Dispatches share the SAME group: the separate dispatch group let a manual
# run execute CONCURRENTLY with an instance-group run (observed live
# 2026-08-11: dispatch 31541231667 co-tenant with pull_request run
# 31530265854, which then wedged past 125m in the suite step) - and
# co-tenancy corrupts whole-queue observation for BOTH runs. The eviction
# pressure that once justified the split is gone now that per-PR triggers
# are removed. One group, one run at a time, for every event.
group: custom-nodes-cloud-instance
Comment thread
coderabbitai[bot] marked this conversation as resolved.
cancel-in-progress: false

jobs:
Expand Down Expand Up @@ -98,6 +105,9 @@ jobs:
(github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository)
runs-on: ubuntu-latest
# Bounded: the longest healthy run observed is ~110m of suite; a wedged
# run otherwise occupies the serial group for GitHub's 360m default.
timeout-minutes: 150
# This proof is intentionally one worker and unsharded. Cloud app boots are
# network-bound, and waitForQueueQuiet observes the whole shared queue.
# Preserve the serial execution contract for the S1-S12 gate.
Expand Down
7 changes: 3 additions & 4 deletions .github/workflows/record-custom-nodes-geometry-cloud.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,9 @@ concurrency:
# queue still makes a record run and a gate run interfere: waitForQueueQuiet
# (customNodeSuite.ts) waits on the whole queue and cannot tell whose work
# it is. Retire this group when the smoke-account pool lands (see the
# gate's concurrency comment for the full semantics).
# EXCEPTION since the gate's dispatch-group split: gate workflow_dispatch
# calibration runs use their own group and CAN overlap this record - do not
# dispatch both at once (manual exclusion until the in-job lock lands).
# gate's concurrency comment for the full semantics). Every gate event -
# including workflow_dispatch - now shares this group, so a record run and
# a gate run can no longer overlap.
group: custom-nodes-cloud-instance
cancel-in-progress: false

Expand Down
Loading