Skip to content

Commit e53f368

Browse files
committed
Follow the trimmed required-secrets mechanism
Re-syncs the vendored engine and re-converges the scheduler workflow, which now carries SCRAPER_API_KEY: ${{ secrets.SCRAPER_API_KEY }} stamped from the two tasks' required_secrets declarations. Delivery is plain workflow env now rather than a parsed bundle, so prose describing the old path is corrected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016NgBmWQmxLFPybrhSTz8i3
1 parent b3c43b1 commit e53f368

13 files changed

Lines changed: 118 additions & 221 deletions

File tree

.claudinite/local/packs/gcec/RULES.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -155,10 +155,10 @@ on that pipeline). Adding a cached live case is the
155155
ScraperAPI's residential proxy (`render=true`, so a single-page app records
156156
post-render HTML with real data). Bot-blocking from CI/sandbox IPs is the
157157
portable rule maintained in the canon; here the escape hatch is the
158-
`SCRAPER_API_KEY` **GitHub Actions secret**, which the scheduler unpacks into a
159-
preprocessing worker's env — never a local fetch (this sandbox is bot-blocked),
160-
and never an agent session (which holds no repo secrets). Both tasks name it in
161-
`required_secrets`, which is how the owner gets asked to configure it. **Never re-introduce a workflow
158+
`SCRAPER_API_KEY` **GitHub Actions secret**. Both tasks name it in
159+
`required_secrets`, which is what puts it in the scheduler workflow's env (and
160+
what gets the owner asked for it) — never a local fetch (this sandbox is
161+
bot-blocked), and never an agent session (which holds no repo secrets). **Never re-introduce a workflow
162162
whose only job is to hold that secret for an agent**: `fetch-page.yml` was
163163
exactly that, and the dispatch/poll/pull round-trip it forced on the agent is
164164
what preprocessing removed. ScraperAPI is the whole fetching surface — swap the

.claudinite/local/packs/gcec/scraperapi.mjs

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,11 @@
77
// ScraperAPI underperforms, and both tasks follow.
88
//
99
// Only usable from a task's `agent_preprocessing` worker: it needs
10-
// SCRAPER_API_KEY, and the scheduler unpacks the repo's Actions secrets into a
11-
// worker's env — an executor session holds none. Both tasks name it in their
12-
// `required_secrets` so the owner is asked to configure it. This sandbox is
13-
// bot-blocked, so a direct fetch from a session would fail even with a key.
10+
// SCRAPER_API_KEY, which both tasks name in `required_secrets` — the wiring
11+
// converge stamps that into the scheduler workflow, so it reaches an Action-side
12+
// worker and nothing else (an executor session holds no repo secrets). This
13+
// sandbox is bot-blocked, so a direct fetch from a session would fail even with
14+
// a key.
1415

1516
import { mkdirSync, writeFileSync, statSync } from 'node:fs';
1617
import { dirname } from 'node:path';

.claudinite/local/packs/gcec/tasks/create-extractor/prepare.mjs

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -16,10 +16,10 @@
1616
// scaffold, prove the offline baseline green, and commit;
1717
// 4. RECORD THE PAGE through ScraperAPI directly. Preprocessing runs inside
1818
// Actions, where SCRAPER_API_KEY already lives (the task names it in
19-
// `required_secrets` so adoption asks the owner to configure it). The
20-
// `fetch-page.yml` workflow existed solely to hold that secret for an agent
21-
// that couldn't — dispatch, poll, pull. Deleted; this is one fetch in the
22-
// same process that needs the bytes;
19+
// `required_secrets`, which the wiring converge stamps into the workflow).
20+
// The `fetch-page.yml` workflow existed solely to hold that secret for an
21+
// agent that couldn't — dispatch, poll, pull. Deleted; this is one fetch in
22+
// the same process that needs the bytes;
2323
// 5. push, open the DRAFT PR the agent continues on, and request the agent
2424
// (CLAUDINITE_REQUEST_AGENT).
2525
//
@@ -229,9 +229,9 @@ export async function main() {
229229
TOKEN = process.env.GITHUB_TOKEN;
230230
const base = process.env.CLAUDINITE_DEFAULT_BRANCH || 'main';
231231
const requestFile = process.env.CLAUDINITE_REQUEST_AGENT;
232-
// Named in task.mjs's `required_secrets`, so the owner has been asked to
233-
// configure it (bootstrap at adoption, a standing issue from the scheduler
234-
// otherwise). Nothing gates on that ask, so check it here and say so plainly.
232+
// Named in task.mjs's `required_secrets`, so the wiring converge put it in this
233+
// process's env and baselining has asked the owner for it if it isn't set.
234+
// Nothing gates on that ask, so check it here and say so plainly.
235235
const scraperKey = process.env.SCRAPER_API_KEY;
236236
if (!REPO || !TOKEN) throw new Error('no CLAUDINITE_REPO/GITHUB_TOKEN — not in an Actions context');
237237
if (!scraperKey) throw new Error('SCRAPER_API_KEY is not set');

.claudinite/local/packs/gcec/tasks/create-extractor/task.mjs

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -37,12 +37,12 @@ export default {
3737
agent_preprocessing_timeout: 1800, // npm ci + the offline suite + a rendered ScraperAPI fetch
3838

3939
// The repo Actions secret this task needs configured (agent-preprocessing
40-
// DESIGN §9) — declarative, so adoption and the scheduler can ask the owner for
41-
// it rather than letting the task fail mysteriously. Being ABLE to read it here
42-
// is what retired `.github/workflows/fetch-page.yml`: that workflow existed only
43-
// because the page fetch needed a secret an agent session cannot hold, so the
44-
// agent had to dispatch it, poll it, and pull its commit. Preprocessing runs
45-
// inside Actions, where the secret already is.
40+
// DESIGN §9). The wiring converge stamps it into the scheduler workflow, so the
41+
// worker reads it as ordinary env; baselining asks the owner if it isn't set.
42+
// Being able to read it at all is what retired `.github/workflows/fetch-page.yml`:
43+
// that workflow existed only because the page fetch needed a secret an agent
44+
// session cannot hold, so the agent had to dispatch it, poll it, and pull its
45+
// commit. Preprocessing runs inside Actions, where the secret already is.
4646
required_secrets: ['SCRAPER_API_KEY'],
4747

4848
// Eligibility, and nothing more. A request is eligible when it is an open

.claudinite/shared/engine/scheduler/converge-wiring.mjs

Lines changed: 33 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -35,13 +35,36 @@ export const CLAUDE_MD = 'CLAUDE.md';
3535
// The whole line (and its trailing newline) is removed wherever it appears.
3636
const CORPUS_IMPORT_RE = /^.*@\.claudinite\/shared\/CLAUDE\.md.*\n?/m;
3737

38+
// The repo Actions secrets its scheduled tasks declare via `required_secrets`,
39+
// deduped and sorted. Async because task discovery is; pure otherwise.
40+
export async function declaredSecrets(root, config) {
41+
const { discoverTasks } = await import('./discover.mjs');
42+
const { tasks } = await discoverTasks(root, config);
43+
return [...new Set(tasks.flatMap((t) => t.decl?.required_secrets ?? []))].sort();
44+
}
45+
46+
// Stamp the declared secrets into the scheduler workflow's engine step, beside
47+
// GITHUB_TOKEN. This is the whole delivery mechanism: GitHub Actions requires each
48+
// secret to be named statically in the workflow, and a task's `required_secrets` is
49+
// exactly that list — so the wiring converge writes it, and a worker then reads
50+
// `process.env.<NAME>` like any other environment variable. No bundle, no parsing,
51+
// no engine-side selection. Regenerated from the stub each converge, so the list
52+
// tracks the declarations rather than accumulating.
53+
export function withDeclaredSecrets(stubText, names = []) {
54+
if (!names.length) return stubText;
55+
const lines = names.map((n) => ` ${n}: \${{ secrets.${n} }}`).join('\n');
56+
return stubText.replace(/^(\s*GITHUB_TOKEN: \$\{\{ github\.token \}\})$/m, `$1\n${lines}`);
57+
}
58+
3859
// Re-converge the scheduler workflow to the vendored stub, with the cron minute set
3960
// to this repo's stable hashed value (never guessed — hash-minute.mjs, a pure
40-
// function of the full name, so re-vendors and this convergence agree). `stubText`
41-
// is the vendored stub's content (the caller reads it from the mount). Returns true
42-
// when the file was written (absent, or drifted from the target).
43-
export function convergeSchedulerWorkflow(root, fullName, stubText) {
44-
const target = stubText.replace(/cron:\s*'[^']*'/, `cron: '${hashedCron(fullName)}'`);
61+
// function of the full name, so re-vendors and this convergence agree) and the
62+
// declared `required_secrets` stamped into the engine step's env. `stubText` is the
63+
// vendored stub's content (the caller reads it from the mount). Returns true when
64+
// the file was written (absent, or drifted from the target).
65+
export function convergeSchedulerWorkflow(root, fullName, stubText, secretNames = []) {
66+
const target = withDeclaredSecrets(stubText, secretNames)
67+
.replace(/cron:\s*'[^']*'/, `cron: '${hashedCron(fullName)}'`);
4568
const path = join(root, SCHEDULER_WORKFLOW);
4669
const current = existsSync(path) ? readFileSync(path, 'utf8') : null;
4770
if (current === target) return false;
@@ -92,9 +115,9 @@ export function removeRetiredCorpusImport(root) {
92115

93116
// Converge every wiring surface, returning a flat summary of what changed (empty
94117
// when the repo was already converged). `stubText` is the vendored scheduler stub.
95-
export function convergeWiring(root, fullName, stubText) {
118+
export function convergeWiring(root, fullName, stubText, secretNames = []) {
96119
const changed = [];
97-
if (convergeSchedulerWorkflow(root, fullName, stubText)) changed.push(SCHEDULER_WORKFLOW);
120+
if (convergeSchedulerWorkflow(root, fullName, stubText, secretNames)) changed.push(SCHEDULER_WORKFLOW);
98121
const hooks = ensureHooks(root);
99122
for (const h of hooks.added) changed.push(`hook:${h}`);
100123
if (removeRetiredCorpusImport(root)) changed.push(`removed retired ${CLAUDE_MD} corpus import`);
@@ -111,7 +134,9 @@ async function main() {
111134
const root = process.env.CLAUDINITE_REPO_ROOT || process.cwd();
112135
const stubPath = join(root, '.claudinite/shared/engine/scheduler/stubs/claudinite-scheduler.yml');
113136
if (!existsSync(stubPath)) { console.error(`converge-wiring: vendored stub not found at ${stubPath}`); process.exit(1); }
114-
const { changed, error } = convergeWiring(root, fullName, readFileSync(stubPath, 'utf8'));
137+
const { loadConfig } = await import('../checks/helpers/repo-context.mjs');
138+
const secretNames = await declaredSecrets(root, loadConfig(root));
139+
const { changed, error } = convergeWiring(root, fullName, readFileSync(stubPath, 'utf8'), secretNames);
115140
if (error) console.log(`! ${error}`);
116141
console.log(changed.length ? `converge-wiring: ${changed.join(', ')}` : 'converge-wiring: already converged');
117142
}

.claudinite/shared/engine/scheduler/preprocess.mjs

Lines changed: 1 addition & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,7 @@
11
// The pre-agent preprocessing stage (agent-preprocessing DESIGN §3). The
22
// scheduler runs a task's declared `agent_preprocessing` command as a SUBPROCESS
33
// before any agent starts — deterministic code work, Action-side, over the one
4-
// sanctioned non-MCP surface (the Action GITHUB_TOKEN, inherited in `env`) and
5-
// the repo's Actions secrets, unpacked into its env (§9, `preprocessingEnv` below).
4+
// sanctioned non-MCP surface (the Action GITHUB_TOKEN, inherited in `env`).
65
//
76
// The subprocess is the scheduler's child, so its `agent_preprocessing_timeout`
87
// is a HARD kill: a manual timer SIGKILLs an overrun and the run is reported
@@ -60,41 +59,6 @@ export function agentRequestPath({ pack, task, slotId }) {
6059
export function clearAgentRequest(path) { try { rmSync(path, { force: true }); } catch { /* nothing to clear */ } }
6160
export function agentRequested(path) { return existsSync(path); }
6261

63-
// --- repo secrets for a worker (agent-preprocessing DESIGN §9) ---------------
64-
// The scheduler workflow hands the engine the repo's secrets as one JSON blob
65-
// (`CLAUDINITE_SECRETS: ${{ toJSON(secrets) }}` — Actions has no way to name
66-
// secrets dynamically). Unpacking it into the worker's env is all there is to
67-
// delivery: a task's `required_secrets` says which ones it NEEDS CONFIGURED (the
68-
// declaration that drives the adoption/scheduler ask), not which ones it is
69-
// permitted to read. Workers are tracked, reviewed code holding the Action
70-
// GITHUB_TOKEN already, so a permission boundary between them would be
71-
// ceremony — the honest boundary is the repo's own secret list.
72-
export const SECRETS_BUNDLE_VAR = 'CLAUDINITE_SECRETS';
73-
74-
// Parse the bundle. Missing/blank/malformed reads as empty — a repo with no
75-
// secrets is the ordinary case, not an error.
76-
export function parseSecretsBundle(raw) {
77-
if (typeof raw !== 'string' || raw.trim() === '') return {};
78-
try {
79-
const parsed = JSON.parse(raw);
80-
return parsed && typeof parsed === 'object' && !Array.isArray(parsed) ? parsed : {};
81-
} catch { return {}; }
82-
}
83-
84-
// The environment a preprocessing subprocess gets: the scheduler's own env with
85-
// the raw bundle REPLACED by the secrets it carried, plus the CLAUDINITE_*
86-
// context. (Replaced, not added alongside — a worker has no use for the blob,
87-
// and leaving it in invites a worker to parse it instead of reading a name.)
88-
// An unset secret renders as "" in `toJSON(secrets)`; those are dropped so a
89-
// worker's `if (!process.env.X)` guard behaves.
90-
export function preprocessingEnv(parentEnv, context) {
91-
const { [SECRETS_BUNDLE_VAR]: bundleRaw, ...rest } = parentEnv;
92-
const secrets = Object.fromEntries(
93-
Object.entries(parseSecretsBundle(bundleRaw)).filter(([, v]) => typeof v === 'string' && v !== ''),
94-
);
95-
return { ...rest, ...secrets, ...context };
96-
}
97-
9862
// A one-line reason for the job summary / an issue comment when preprocessing
9963
// fails — distinguishing a timeout kill from a non-zero exit.
10064
export function preprocessingFailure(result) {

.claudinite/shared/engine/scheduler/required-secrets.mjs

Lines changed: 0 additions & 83 deletions
This file was deleted.

0 commit comments

Comments
 (0)