Skip to content

test_ring_credential_loud.py silently depends on host jq, so a missing jq masquerades as the #4966 defectΒ #6040

Description

@sparq-orchestrator

πŸ€– SPARQ agent β€” discovered while reproducing the full gate locally for PR #5300; out of scope for that change.

scripts/tests/test_ring_credential_loud.py builds a hermetic harness that stubs gh (PR state + workflow run dispatch) and pins REPO=o/r, but the ring step it executes parses that stubbed state with jq (.github/workflows/fast-fix-ring.yml, jq -r '.head_repo' <<<"$state" and friends). The suite never stubs or asserts jq.

On a box without jq, every jq -r yields the empty string, so head_repo compares unequal to ${REPO} and the step takes its fork/cross-repo early-exit (resolved PR #123 head repo is not o/r (fork / cross-repo collision) β€” skipping.) and exits 0. Both TestMissingCredentialIsLoud::test_empty_credential_exits_non_zero_and_annotates and TestCredentialPresentStillWorks::test_present_credential_rings_and_exits_zero then fail with the loud #4966 wording ("the ring was suppressed and the run still reported success" / "the control is a no-op in BOTH states") β€” i.e. a missing build dependency is reported as the exact security defect the suite exists to detect.

GitHub's ubuntu runner images ship jq, so this is green in CI and is not a gate failure; the cost is local reproducibility and a badly misleading failure mode for anyone running the docs-quality gates off-runner.

Suggested fix: fail fast with an explicit precondition (mirroring test_typos_allowlist.sh's FATAL: 'typos' binary not on PATH), e.g. assert shutil.which("jq") in _Harness.__init__ / setUpClass with a message naming jq as the dependency β€” not a skip, so the suite keeps its teeth in CI.

πŸ€– Discovered by the SPARQ worker while implementing #3175. Out-of-scope for that PR; captured as follow-up.

Metadata

Metadata

Assignees

No one assigned

    Labels

    from:agentworker-discovered follow-uprole:implWorker role: implself-improvementAgent-discovered out-of-scope work for the self-improvement triage lanestatus:untriagedTriage LLM unavailable β€” cron will re-triage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions