Skip to content

Commit ba57f85

Browse files
bpamiriclaude
bpamiri
and
claude
authored
ci: non-dev environment smoke probes (production/testing matrix) (#3044)
* ci: environment smoke probes for non-dev request behavior Five probes runnable against any live instance via BASE_URL/SMOKE_ENV: root renders without 5xx or trace leakage, /wheels/info aborts to a clean 404 outside development (pins issue #3029's cfabort class), unknown routes 404 without stack traces, and unauthenticated/wrong-password reloads are refused (no 302). Red-verified against pre-fix develop: Adobe 2023 in testing mode fails the root and wheels-info probes exactly as issue #3029 predicts; Lucee 7 control passes all five. Refs #3029 #3030 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Peter Amiri <peter@alurium.com> * ci: smoke-env hardening — connectivity preflight, explicit 000 failures, mktemp Quality-review fixes: unreachable-server status 000 no longer satisfies the NOT-302 / non-5xx assertions vacuously (upfront connectivity probe + explicit 000 failure branches in the reload probes); probe 1 uses mktemp instead of a fixed /tmp path; curls are -sS with --connect-timeout 5; Adobe variable-name marker matches digits/underscores; SMOKE_ENV header comment states it is a log label only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Peter Amiri <peter@alurium.com> * ci: production/testing-mode smoke job running the environment probes Boots the demo app with config/environment.cfm forced to production and testing (matrix) and runs tools/ci/smoke-env.sh. Green-verified live on Lucee 7 and Adobe 2023 in both environments on post-fix develop; the same probes fail on the pre-fix tree (Adobe 500s from issue #3029), so this job would have caught discussion #3023 the day the bug landed. Refs #3029 #3030 #3031 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Peter Amiri <peter@alurium.com> * ci: smoke-env review nits — trigger on lucee.ci.json, assert the env sed applied The workflow copies tools/ci/lucee.ci.json as the live server config, so changes to it must trigger the job; and the post-sed grep now fails the step when the substitution no-ops (pattern drift would otherwise smoke- test development mode with misleading probe failures). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Peter Amiri <peter@alurium.com> * ci: smoke-env via the setup-wheels-test-env composite, honest Lucee-only scope Review findings: the header claimed this job nets issue #3029, but that class is Adobe-only and this job boots Lucee 7 only — by the PR's own red-verification data the Lucee leg stays green on the pre-fix tree. The header now states the real coverage (#3030/#3031 + engine-agnostic non-dev regressions), points at BareCfabortGuardSpec for the cfabort idiom, and the Adobe smoke leg is tracked in #3047. Also replaces the ~100-line boot block copied from pr.yml with the existing setup-wheels-test-env composite action (install-playwright: false) — no third copy of the LUCLI_VERSION/JDBC bump surface. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Peter Amiri <peter@alurium.com> * ci: smoke-env paths filter includes the composite action it boots with Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Peter Amiri <peter@alurium.com> --------- Signed-off-by: Peter Amiri <peter@alurium.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent a6df6be commit ba57f85

2 files changed

Lines changed: 163 additions & 0 deletions

File tree

‎.github/workflows/smoke-env.yml‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
name: Smoke (non-dev environments)
2+
3+
# Lucee-leg regression net for discussion #3023 / issues #3030, #3031 (and any
4+
# future engine-agnostic non-dev request regression): boots the demo app with
5+
# config/environment.cfm forced to production and testing and asserts
6+
# non-development request behavior (clean 404s, no stack traces, reload refused
7+
# without the password) via tools/ci/smoke-env.sh.
8+
#
9+
# Scope note: this job runs Lucee 7 only. The Adobe-only failure class from
10+
# issue #3029 (engine error-template leakage in non-dev environments) is NOT
11+
# gated here — the cfabort idiom itself is pinned structurally by
12+
# vendor/wheels/tests/specs/security/BareCfabortGuardSpec.cfc, and the Adobe
13+
# smoke leg is tracked in issue #3047.
14+
15+
on:
16+
pull_request:
17+
branches:
18+
- develop
19+
paths:
20+
- 'vendor/wheels/**'
21+
- 'public/**'
22+
- 'config/**'
23+
- 'app/**'
24+
- 'tools/ci/smoke-env.sh'
25+
- '.github/workflows/smoke-env.yml'
26+
- 'tools/ci/lucee.ci.json'
27+
- '.github/actions/setup-wheels-test-env/**'
28+
29+
permissions:
30+
contents: read
31+
32+
jobs:
33+
smoke:
34+
name: "Smoke: ${{ matrix.wheels_env }} (Lucee 7)"
35+
runs-on: ubuntu-latest
36+
timeout-minutes: 20
37+
strategy:
38+
fail-fast: false
39+
matrix:
40+
wheels_env: [production, testing]
41+
env:
42+
WHEELS_CI: "true"
43+
steps:
44+
- uses: actions/checkout@v5
45+
46+
# Must run before the server boots: the app cold-boots directly into the
47+
# matrix environment (in-place reload does not re-read environment.cfm).
48+
- name: Force ${{ matrix.wheels_env }} environment
49+
run: |
50+
sed -i 's/set(environment = "[a-z]*")/set(environment = "${{ matrix.wheels_env }}")/' config/environment.cfm
51+
grep -qF 'set(environment = "${{ matrix.wheels_env }}")' config/environment.cfm || {
52+
echo "::error::environment.cfm substitution failed — pattern drift?"; exit 1; }
53+
grep -n 'set(environment' config/environment.cfm
54+
55+
# Canonical boot harness (JDK 21, LuCLI, SQLite DBs, JDBC, server start +
56+
# restart) — the same composite the bot workflows use, instead of a third
57+
# inline copy of pr.yml's boot block. Probes are plain curl, no browser.
58+
- name: Set up Wheels test environment
59+
uses: ./.github/actions/setup-wheels-test-env
60+
with:
61+
install-playwright: 'false'
62+
63+
- name: Run smoke probes
64+
run: |
65+
BASE_URL="http://localhost:60007" SMOKE_ENV="${{ matrix.wheels_env }}" bash tools/ci/smoke-env.sh
66+
67+
- name: Debug server logs
68+
if: failure()
69+
run: cat /tmp/lucli-server.log 2>/dev/null || true
70+
71+
- name: Stop server
72+
if: always()
73+
run: |
74+
if [ -f /tmp/lucli-server.pid ]; then
75+
kill $(cat /tmp/lucli-server.pid) 2>/dev/null || true
76+
fi

‎tools/ci/smoke-env.sh‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
#!/usr/bin/env bash
2+
# Environment-mode smoke probes (issue #3023 regression net).
3+
# Boots nothing itself: point it at an already-running app via BASE_URL.
4+
# SMOKE_ENV is a log label only; all probes assume a non-development environment.
5+
#
6+
# BASE_URL=http://localhost:60007 SMOKE_ENV=production bash tools/ci/smoke-env.sh
7+
#
8+
# Exit 0 = all probes pass. Non-zero = at least one probe failed; every
9+
# failure prints PROBE/EXPECTED/GOT lines for the CI log.
10+
set -u
11+
BASE_URL="${BASE_URL:-http://localhost:60007}"
12+
SMOKE_ENV="${SMOKE_ENV:-production}"
13+
FAILURES=0
14+
15+
# Markers that must NEVER appear in any non-development response body:
16+
# engine stack traces, raw CFML errors, debug output, the generic error
17+
# template leaking on routes that should be plain 404s.
18+
TRACE_MARKERS='coldfusion\.runtime|lucee\.runtime|Variable [A-Z][A-Z0-9_]* is undefined|Error Occurred While Processing Request|<!-- wheels-debug|id="wheels-debugbar"'
19+
20+
# Fail fast and unambiguously when the app is unreachable — without this,
21+
# curl status 000 satisfies the NOT-302 and non-5xx assertions vacuously.
22+
if ! curl -sS -o /dev/null --connect-timeout 5 --max-time 15 "$BASE_URL/"; then
23+
echo "FAIL probe=connectivity url=$BASE_URL/ (server unreachable; skipping probes)"
24+
exit 1
25+
fi
26+
27+
probe() { # name url expected_status body_must_match body_must_not_match
28+
local name="$1" url="$2" expect="$3" must="$4" mustnot="$5"
29+
local body status
30+
body=$(mktemp)
31+
status=$(curl -sS -o "$body" -w '%{http_code}' --connect-timeout 5 --max-time 60 "$url")
32+
local ok=1
33+
[ "$status" = "$expect" ] || ok=0
34+
if [ -n "$must" ] && ! grep -qE "$must" "$body"; then ok=0; fi
35+
if [ -n "$mustnot" ] && grep -qE "$mustnot" "$body"; then ok=0; fi
36+
if [ "$ok" = "0" ]; then
37+
echo "FAIL probe=$name url=$url"
38+
echo " expected: status=$expect must=~'$must' mustnot=~'$mustnot'"
39+
echo " got: status=$status body_head=$(head -c 200 "$body" | tr '\n' ' ')"
40+
FAILURES=$((FAILURES+1))
41+
else
42+
echo "PASS probe=$name ($status)"
43+
fi
44+
rm -f "$body"
45+
}
46+
47+
# 1. Root route renders without server error and without debug/trace leakage.
48+
# (Any 2xx/3xx/404 is acceptable app behavior; 5xx is not.)
49+
root_body=$(mktemp)
50+
status=$(curl -sS -o "$root_body" -w '%{http_code}' --connect-timeout 5 --max-time 60 "$BASE_URL/")
51+
case "$status" in
52+
5*) echo "FAIL probe=root-no-5xx url=$BASE_URL/ got status=$status"; FAILURES=$((FAILURES+1));;
53+
000) echo "FAIL probe=root-no-5xx url=$BASE_URL/ (status=000, request failed)"; FAILURES=$((FAILURES+1));;
54+
*) if grep -qE "$TRACE_MARKERS" "$root_body"; then
55+
echo "FAIL probe=root-no-trace-markers url=$BASE_URL/ (status=$status)"; FAILURES=$((FAILURES+1))
56+
else
57+
echo "PASS probe=root ($status, clean)"
58+
fi;;
59+
esac
60+
rm -f "$root_body"
61+
62+
# 2. Public component is OFF outside development and aborts CLEANLY (issue #3029):
63+
# plain 404 + "Not Found", no error template, no stack trace, on every engine.
64+
probe "wheels-info-clean-404" "$BASE_URL/wheels/info" "404" "Not Found" "$TRACE_MARKERS|Something went wrong"
65+
66+
# 3. Unknown route returns the application 404 path, not a 500/stack trace.
67+
probe "unknown-route-404" "$BASE_URL/smoke-nonexistent-route-$$" "404" "" "$TRACE_MARKERS"
68+
69+
# 4. Reload without a password must be refused: the reload path responds 302
70+
# (applicationStop + redirect); a refusal renders normally. Assert NOT 302.
71+
status=$(curl -sS -o /dev/null -w '%{http_code}' --connect-timeout 5 --max-time 60 "$BASE_URL/?reload=true")
72+
case "$status" in
73+
302) echo "FAIL probe=reload-unauthenticated-refused url=$BASE_URL/?reload=true (got 302 = reload executed without password)"; FAILURES=$((FAILURES+1));;
74+
000) echo "FAIL probe=reload-unauthenticated-refused (status=000, request failed)"; FAILURES=$((FAILURES+1));;
75+
*) echo "PASS probe=reload-unauthenticated-refused ($status)";;
76+
esac
77+
78+
# 5. Wrong password must also be refused.
79+
status=$(curl -sS -o /dev/null -w '%{http_code}' --connect-timeout 5 --max-time 60 "$BASE_URL/?reload=true&password=definitely-wrong-$$")
80+
case "$status" in
81+
302) echo "FAIL probe=reload-wrong-password-refused url=$BASE_URL/?reload=true&password=... (got 302)"; FAILURES=$((FAILURES+1));;
82+
000) echo "FAIL probe=reload-wrong-password-refused (status=000, request failed)"; FAILURES=$((FAILURES+1));;
83+
*) echo "PASS probe=reload-wrong-password-refused ($status)";;
84+
esac
85+
86+
echo "smoke-env: env=$SMOKE_ENV failures=$FAILURES"
87+
exit $FAILURES

0 commit comments

Comments
 (0)