Skip to content

Commit cccfaac

Browse files
castrojoCopilot
andcommitted
ci: remove remote execution configuration
Assisted-by: GitHub Copilot Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 5474540 commit cccfaac

6 files changed

Lines changed: 5 additions & 107 deletions

File tree

.github/actions/generate-bst-ci-config/action.yml

Lines changed: 3 additions & 99 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,7 @@
11
name: Generate BuildStream CI config
2-
description: Write buildstream-ci.conf for CI builds and optional remote execution.
2+
description: Write BuildStream cache configuration for runner-side assembly.
33

44
inputs:
5-
enable-remote-execution:
6-
description: Enable x86_64 remote execution settings and max-jobs tuning.
7-
required: true
85
enable-push:
96
description: Push artifacts and sources to the remote cache. Set false for fetch-only jobs (e.g. publish).
107
required: false
@@ -16,7 +13,6 @@ runs:
1613
- name: Generate BuildStream CI config
1714
shell: bash
1815
env:
19-
ENABLE_REMOTE_EXECUTION: ${{ inputs.enable-remote-execution }}
2016
ENABLE_PUSH: ${{ inputs.enable-push }}
2117
run: |
2218
mkdir -p logs
@@ -39,36 +35,11 @@ runs:
3935
logdir: /src/logs
4036
BSTCONF
4137
42-
if [[ -n "${CASD_CLIENT_CERT:-}" ]] && [[ -n "${CASD_CLIENT_KEY:-}" ]]; then
43-
if [[ "$ENABLE_REMOTE_EXECUTION" == "true" ]]; then
44-
cat >> buildstream-ci.conf <<'BSTCONFBUILD'
45-
build:
46-
retry-failed: True
47-
# max-jobs does NOT affect cache keys (buildelement.py: "normally
48-
# automatically resolved and does not affect the cache key"), so
49-
# tuning it preserves the warm CAS. History: max-jobs: 1 made every
50-
# big element (llvm, WebKit x2) a 10-30h single-core compile that
51-
# mathematically busted the 480m budget (11-day outage). The first
52-
# attempt at 16 coincided with a server-wide outage (RST_STREAM
53-
# INTERNAL + FetchBlob DEADLINE_EXCEEDED on all pulls, 23:20 UTC
54-
# 2026-07-09) — likely OOM: gimple-match.cc compiles eat 4-6 GB
55-
# each, and 16 in parallel plus casd cache pressure exceeds the
56-
# 128 GB builder. 2 builders x -j8 is a safe 8x speedup over -j1
57-
# while staying well under the RAM ceiling.
58-
max-jobs: 8
59-
BSTCONFBUILD
60-
else
61-
cat >> buildstream-ci.conf <<'BSTCONFBUILD'
62-
build:
63-
retry-failed: True
64-
BSTCONFBUILD
65-
fi
66-
else
67-
cat >> buildstream-ci.conf <<'BSTCONFBUILD'
38+
cat >> buildstream-ci.conf <<'BSTCONFBUILD'
6839
build:
6940
retry-failed: True
41+
max-jobs: 1
7042
BSTCONFBUILD
71-
fi
7243
7344
if [[ -n "${CASD_CLIENT_CERT:-}" ]] && [[ -n "${CASD_CLIENT_KEY:-}" ]]; then
7445
PUSH_FLAG="true"
@@ -130,56 +101,6 @@ runs:
130101
131102
BSTCONFCACHE
132103
133-
# Top-level storage-service (under cache:) is ONLY for enable-push: true.
134-
# Remote execution uses the NESTED storage-service inside remote-execution: block.
135-
# NEVER add top-level storage-service when only RE is enabled — it puts casd
136-
# in proxy mode and causes gRPC flooding. See docs/skills/ci.md.
137-
138-
if [[ "$ENABLE_REMOTE_EXECUTION" == "true" ]]; then
139-
# IMPORTANT: storage-service is NESTED inside remote-execution, NOT at
140-
# the top-level cache: block. This satisfies BST's validation requirement
141-
# ("Remote execution requires storage-service") without putting buildbox-casd
142-
# into write-through proxy mode (--cas-remote). Top-level cache.storage-service
143-
# routes ALL casd operations through the remote and causes gRPC connection drops
144-
# after ~3.5 hours. The nested form only affects per-action RE blob transfers
145-
# (transient connections), avoiding the sustained stream that killed builds.
146-
# See docs/skills/ci.md "Remote Execution with Nested storage-service".
147-
cat >> buildstream-ci.conf <<'BSTCONFREMOTE'
148-
remote-execution:
149-
execution-service:
150-
url: https://cache.projectbluefin.io:11002
151-
connection-config:
152-
keepalive-time: 180
153-
retry-limit: 5
154-
retry-delay: 1000
155-
request-timeout: 180
156-
auth:
157-
client-key: /src/client.key
158-
client-cert: /src/client.crt
159-
storage-service:
160-
url: https://cache.projectbluefin.io:11002
161-
connection-config:
162-
keepalive-time: 180
163-
retry-limit: 5
164-
retry-delay: 1000
165-
request-timeout: 180
166-
auth:
167-
client-key: /src/client.key
168-
client-cert: /src/client.crt
169-
action-cache-service:
170-
url: https://cache.projectbluefin.io:11002
171-
connection-config:
172-
keepalive-time: 180
173-
retry-limit: 5
174-
retry-delay: 1000
175-
request-timeout: 180
176-
auth:
177-
client-key: /src/client.key
178-
client-cert: /src/client.crt
179-
BSTCONFREMOTE
180-
fi
181-
fi
182-
183104
# Write the unified cache configuration block to avoid duplicate keys.
184105
cat >> buildstream-ci.conf <<'BSTCONFCACHEBLOCK'
185106
@@ -204,24 +125,7 @@ runs:
204125
fi
205126
fi
206127
207-
# Preserve the generated config in the uploaded logs artifact so the next run
208-
# can confirm the remote-execution block was emitted and not silently skipped.
209128
cp buildstream-ci.conf logs/buildstream-ci.conf
210-
211-
if [[ "$ENABLE_REMOTE_EXECUTION" == "true" ]]; then
212-
if [[ -z "${CASD_CLIENT_CERT:-}" ]] || [[ -z "${CASD_CLIENT_KEY:-}" ]]; then
213-
echo "::error::remote execution was requested but CASD client credentials are missing"
214-
exit 1
215-
fi
216-
if ! grep -Fq 'remote-execution:' buildstream-ci.conf; then
217-
echo "::error::remote-execution block missing from generated buildstream-ci.conf"
218-
exit 1
219-
fi
220-
echo "Remote execution enabled: yes"
221-
else
222-
echo "Remote execution enabled: no"
223-
fi
224-
225129
echo "=== BuildStream CI config ==="
226130
cat buildstream-ci.conf
227131

.github/workflows/build-aarch64.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -97,7 +97,7 @@ jobs:
9797
CASD_CLIENT_KEY: ${{ secrets.CASD_CLIENT_KEY }}
9898
uses: ./.github/actions/generate-bst-ci-config
9999
with:
100-
enable-remote-execution: 'false'
100+
101101
enable-push: 'true'
102102

103103
- name: Build OCI image with BuildStream (aarch64)

.github/workflows/build.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,6 @@ jobs:
119119
# Remote execution is disabled here so we pull cached artifacts from the CAS
120120
# and avoid the remote execution queue and server capacity bottlenecks.
121121
# We perform final image assembly locally on the GHA runner.
122-
enable-remote-execution: "false"
123122
enable-push: "false"
124123

125124
# Count the full element graph so the progress script can show %.

.github/workflows/publish.yml

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -134,7 +134,6 @@ jobs:
134134
CASD_CLIENT_KEY: ${{ secrets.CASD_CLIENT_KEY }}
135135
uses: ./.github/actions/generate-bst-ci-config
136136
with:
137-
enable-remote-execution: 'false'
138137
enable-push: 'false'
139138

140139
- name: Export OCI image from BuildStream
@@ -577,7 +576,6 @@ jobs:
577576
CASD_CLIENT_KEY: ${{ secrets.CASD_CLIENT_KEY }}
578577
uses: ./.github/actions/generate-bst-ci-config
579578
with:
580-
enable-remote-execution: 'false'
581579
enable-push: 'false'
582580

583581
# Cache the pip wheel for buildstream-sbom across runs.

.github/workflows/update-filemap.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,6 @@ jobs:
5252
- name: Generate BuildStream CI config
5353
uses: ./.github/actions/generate-bst-ci-config
5454
with:
55-
enable-remote-execution: 'false'
5655
enable-push: 'false'
5756
env:
5857
CASD_CLIENT_CERT: ${{ vars.CASD_CLIENT_CERT }}

docs/skills/ci.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,7 @@ This is the fast path for stale-image complaints: the image date is usually wron
107107

108108
| Workflow | Trigger | What it does |
109109
|---|---|---|
110-
| `build.yml` | `push: testing` for key-busting paths, `workflow_dispatch`, `schedule: daily 13:00 UTC` — NOT `pull_request` or `merge_group` | Pull-only warmup shards → serialized CAS pushes → BST build → artifacts into remote CAS. Does NOT push to GHCR. |
110+
| `build.yml` | `push: testing` for key-busting paths, `workflow_dispatch`, `schedule: daily 13:00 UTC` — NOT `pull_request` or `merge_group` | Cache-only final OCI assembly → artifact cache. Does NOT push to GHCR. |
111111
| `publish.yml` | `workflow_run` from `build.yml` (branches: testing, next, + their gh-readonly-queue/* paths) | Export from CAS → push `:$sha` → sign/attest → promote to `:testing`/`:next`. No build happens here. |
112112
| `execute-release.yml` | `workflow_run` from `publish.yml` on `testing`, `workflow_dispatch` | SHA freshness check (:testing vs :stable). If different: cosign verify → skopeo copy `:testing``:stable` → fast-forward main → create GitHub Release. Skips if equal. (Boot-check is in publish.yml before :testing; execute-release trusts the already-boot-checked image.) |
113113
| ~~`promote-testing-to-main.yml`~~ | DELETED | Was: `push: testing`, schedule Tue 04:00 UTC, manual. |
@@ -158,8 +158,6 @@ is disabled.
158158
are present. The protected `testing` workflow requires these credentials; without
159159
them, do not run a source-build fallback.
160160

161-
**Remote-execution sanity check:** when `enable-remote-execution: 'true'`, the generated `buildstream-ci.conf` must contain a real `remote-execution:` block. The workflow now stores the generated config in the `logs/` artifact and fails fast if the block is missing or the remote cache credentials are absent.
162-
163161
## ⚠️ Pre-Commit BST Syntax Gate
164162

165163
For any change to `project.conf`, `*.bst` elements, or `Justfile`:

0 commit comments

Comments
 (0)