-
Notifications
You must be signed in to change notification settings - Fork 4
164 lines (145 loc) · 8.17 KB
/
Copy pathpi-upgrade-check.yml
File metadata and controls
164 lines (145 loc) · 8.17 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
# REQ-UPSTREAM-CONTRACT-TESTS — the pi upgrade gate.
#
# Asserts the assumptions pi-dispatch pins about pi, and fails the build when one stops holding.
# CONST-PI-VERSION-PINNED makes an upgrade an explicit commit that edits a version string, so this
# workflow is what turns that commit into a gate rather than a hope.
#
# Every assertion below covers a failure that is SILENT. A crash reports itself and needs no test;
# these do not. Each maps to a point where a claim about pi was verified, believed, and later refuted:
# - "appendSystemPrompt composes with file discovery" -> it REPLACES it, via a `??`
# - "createAgentSession reloads the loader you pass" -> only one it built itself
# - "-nc is a flag you remember" -> it is an object you forget to build
# - "pi never throws" -> its own JSDoc says otherwise
# Each was found by reading source, not by running code. These tests are what catch the next one.
name: pi upgrade check
on:
push:
paths:
- "image/**"
- "guardrails/**"
- "package.json"
- ".github/workflows/pi-upgrade-check.yml"
pull_request:
paths:
- "image/**"
- "guardrails/**"
- "package.json"
- ".github/workflows/pi-upgrade-check.yml"
schedule:
# Weekly: pi ships breaking changes between minors and its HEAD moved within 24h of this
# project's design being written. A pin that is never exercised rots silently.
- cron: "0 6 * * 1"
workflow_dispatch:
jobs:
version-pin:
name: pins are exact (CONST-PI-VERSION-PINNED)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: No floating ranges on pi packages
run: |
# A floating range turns a silent upstream minor into every queued job becoming a
# no-op with no signal, because the queue still reports success.
if grep -rnE '@earendil-works/pi-[a-z-]+@[\^~]|"@earendil-works/pi-[a-z-]+":[[:space:]]*"[\^~]' image/ package.json; then
echo "::error::Floating range on a pi package. CONST-PI-VERSION-PINNED requires an exact pin."
exit 1
fi
echo "OK: pi pinned exactly"
- name: Base image is pinned by digest
run: |
# Same reasoning, different vendor. A floating base tag is the identical silent break.
if ! grep -qE '^FROM .*@sha256:[0-9a-f]{64}' image/Dockerfile; then
echo "::error::Base image is not digest-pinned in image/Dockerfile."
exit 1
fi
echo "OK: base pinned by digest"
contract-tests:
name: pinned assumptions still hold (offline, no API key)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version-file: ".nvmrc" # pi requires >=22.19.0; engine-strict enforces it
- run: npm ci --no-audit --no-fund
# The scariest assertions in this project are FREE: every trap lives at the resource-loader
# boundary (pure) or at before_agent_start (which fires strictly before any provider HTTP
# call). No API key, no tokens, no flake, no excuse not to run them on every build.
#
# PI_DISPATCH_REQUIRE_LOADER_TESTS=1 turns a skip into a hard failure. A skipped assertion is
# an UNVERIFIED assertion, and "skipped = pass" is precisely the reasoning that lets a
# guardrail-less agent ship green.
- name: Guardrails present, hostile AGENTS.md absent, exit codes correct
env:
PI_DISPATCH_REQUIRE_LOADER_TESTS: "1"
run: npm test
image:
name: the job image holds its contract
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Build
run: docker build -f image/Dockerfile -t pi-job:ci .
# --- INT-SDK-SESSION-OPTIONS ---
# `pi --mode print` does not exist: --mode accepts text|json|rpc, and --print/-p is a separate
# boolean. --help rather than a real prompt, because a real prompt needs a paid API key and a
# contract test that costs money is a contract test that gets disabled.
# --entrypoint is required: ENTRYPOINT is the runner and ignores CMD, so `docker run img pi ...`
# would silently run the runner instead of pi and the assertion would test nothing.
- name: pi -p is still a flag
run: docker run --rm --entrypoint pi pi-job:ci -p --help >/dev/null
# --- CONST-ISOLATION-CONTAINER-PER-JOB ---
# --cap-drop=ALL is the enforcement surface. Read the effective capability set directly rather
# than install libcap just to ask.
- name: No capabilities under --cap-drop=ALL
run: |
caps=$(docker run --rm --cap-drop=ALL --security-opt no-new-privileges \
--entrypoint sh pi-job:ci -c 'grep ^CapEff /proc/self/status' | awk '{print $2}')
[ "$caps" = "0000000000000000" ] || { echo "::error::Container retains capabilities: $caps"; exit 1; }
echo "OK: CapEff=$caps"
- name: Runs as a non-root user
run: |
uid=$(docker run --rm --entrypoint id pi-job:ci -u)
[ "$uid" != "0" ] || { echo "::error::Job container runs as root."; exit 1; }
# --- INT-CONTAINER-JOB-INPUTS ---
# /job:ro is what makes CONST-ISSUE-TEXT-IS-DATA enforceable by filesystem permission rather
# than by asking nicely. It is a security boundary, so assert the kernel enforces it.
- name: /job is genuinely read-only from inside
run: |
mkdir -p fixture/pi
echo "x" > fixture/pi/APPEND_SYSTEM.md
if docker run --rm --cap-drop=ALL -v "$PWD/fixture:/job:ro" --entrypoint sh pi-job:ci \
-c 'echo pwned > /job/pi/APPEND_SYSTEM.md' 2>/dev/null; then
echo "::error::/job is writable. The agent can rewrite its own instructions."
exit 1
fi
echo "OK: /job:ro enforced"
# --- INT-CONTAINER-RUNTIME-CONTRACT ---
# pi lazily creates ~/.pi/agent and writes auth.json on the FIRST credential operation. If the
# dir is root-owned the job dies EACCES at runtime, on a path nothing in the Dockerfile hints
# at. COPY --chown does not fix it (it skips auto-created parents), so assert the real thing.
- name: The agent dir is writable by the runtime user
run: docker run --rm --entrypoint sh pi-job:ci -c 'touch "$HOME/.pi/agent/auth.json" && rm "$HOME/.pi/agent/auth.json"'
- name: The guardrails are baked where the runner reads them
run: |
docker run --rm --entrypoint grep pi-job:ci -q "pi-dispatch-guardrails-v1" /opt/pi-dispatch/HARD_RULES.md \
|| { echo "::error::Guardrails sentinel missing from /opt/pi-dispatch/HARD_RULES.md"; exit 1; }
# --- DES-PLAYWRIGHT-CLI-NOT-CHROME-DEVTOOLS / REQ-FRONTEND-VISUAL-VERIFY ---
# Root-installed Chromium lands in /root/.cache/ms-playwright, invisible to the non-root user
# CONST-ISOLATION-CONTAINER-PER-JOB requires. --shm-size, NOT --ipc=host: Playwright recommends
# the latter, but it shares the HOST IPC namespace with a container running adversarial-input
# agent code. The crash it prevents is caused by a 64MB /dev/shm, so fix that instead.
- name: Chromium screenshots a local page as non-root
run: |
echo '<html><body><h1>pi-dispatch</h1></body></html>' > fixture/page.html
docker run --rm --init --cap-drop=ALL --security-opt no-new-privileges --shm-size=1g \
-v "$PWD/fixture:/fixture:ro" --entrypoint sh pi-job:ci \
-c 'playwright-cli screenshot --browser-arg=--no-sandbox file:///fixture/page.html /tmp/x.png && test -s /tmp/x.png' \
|| { echo "::error::Chromium unusable as non-root — check PLAYWRIGHT_BROWSERS_PATH at build AND run"; exit 1; }
# Fonts absent => tofu boxes => screenshots that look fine and contain no legible text. That
# silently guts the requirement full Chromium is in this image for.
- name: Fonts are installed (or screenshots are tofu)
run: |
n=$(docker run --rm --entrypoint sh pi-job:ci -c 'fc-list | wc -l')
[ "$n" -gt 0 ] || { echo "::error::No fonts. Chromium will render boxes and REQ-FRONTEND-VISUAL-VERIFY is a lie."; exit 1; }
echo "OK: $n fonts"