π€ SPARQ agent β discovered while reproducing the full gate locally for PR #5300; out of scope for that change.
scripts/tests/test_ring_credential_loud.py builds a hermetic harness that stubs gh (PR state + workflow run dispatch) and pins REPO=o/r, but the ring step it executes parses that stubbed state with jq (.github/workflows/fast-fix-ring.yml, jq -r '.head_repo' <<<"$state" and friends). The suite never stubs or asserts jq.
On a box without jq, every jq -r yields the empty string, so head_repo compares unequal to ${REPO} and the step takes its fork/cross-repo early-exit (resolved PR #123 head repo is not o/r (fork / cross-repo collision) β skipping.) and exits 0. Both TestMissingCredentialIsLoud::test_empty_credential_exits_non_zero_and_annotates and TestCredentialPresentStillWorks::test_present_credential_rings_and_exits_zero then fail with the loud #4966 wording ("the ring was suppressed and the run still reported success" / "the control is a no-op in BOTH states") β i.e. a missing build dependency is reported as the exact security defect the suite exists to detect.
GitHub's ubuntu runner images ship jq, so this is green in CI and is not a gate failure; the cost is local reproducibility and a badly misleading failure mode for anyone running the docs-quality gates off-runner.
Suggested fix: fail fast with an explicit precondition (mirroring test_typos_allowlist.sh's FATAL: 'typos' binary not on PATH), e.g. assert shutil.which("jq") in _Harness.__init__ / setUpClass with a message naming jq as the dependency β not a skip, so the suite keeps its teeth in CI.
π€ Discovered by the SPARQ worker while implementing #3175. Out-of-scope for that PR; captured as follow-up.
scripts/tests/test_ring_credential_loud.pybuilds a hermetic harness that stubsgh(PR state +workflow rundispatch) and pinsREPO=o/r, but the ring step it executes parses that stubbed state withjq(.github/workflows/fast-fix-ring.yml,jq -r '.head_repo' <<<"$state"and friends). The suite never stubs or assertsjq.On a box without
jq, everyjq -ryields the empty string, sohead_repocompares unequal to${REPO}and the step takes its fork/cross-repo early-exit (resolved PR #123 head repo is not o/r (fork / cross-repo collision) β skipping.) and exits 0. BothTestMissingCredentialIsLoud::test_empty_credential_exits_non_zero_and_annotatesandTestCredentialPresentStillWorks::test_present_credential_rings_and_exits_zerothen fail with the loud #4966 wording ("the ring was suppressed and the run still reported success" / "the control is a no-op in BOTH states") β i.e. a missing build dependency is reported as the exact security defect the suite exists to detect.GitHub's ubuntu runner images ship
jq, so this is green in CI and is not a gate failure; the cost is local reproducibility and a badly misleading failure mode for anyone running the docs-quality gates off-runner.Suggested fix: fail fast with an explicit precondition (mirroring
test_typos_allowlist.sh'sFATAL: 'typos' binary not on PATH), e.g. assertshutil.which("jq")in_Harness.__init__/setUpClasswith a message namingjqas the dependency β not a skip, so the suite keeps its teeth in CI.