Add quality workflow OS and claim safety gate - #67
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6b6dc21a58
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| - This document does **not** assert that LiouScope is production-ready. | ||
| - This document does **not** assert that LiouScope is externally certified. | ||
| - This document does **not** assert that PyPI or DOI publication is complete. |
There was a problem hiding this comment.
The new Quality Contract workflow runs python .github/scripts/check_claim_safety.py, but these newly added negative disclaimers are rejected because the checker only allows literal markers like not / does not while the Markdown source has does **not**. Running the new check on this commit reports these three lines and exits 1, so every PR/push that runs this workflow will be red until the wording or matcher is fixed.
Useful? React with 👍 / 👎.
| ROOT = Path(__file__).resolve().parents[2] | ||
| WORKFLOWS = ROOT / ".github" / "workflows" | ||
| FULL_SHA_RE = re.compile(r"^[0-9a-f]{40}$", re.IGNORECASE) | ||
| USES_RE = re.compile(r"^\s*uses:\s*([^\s#]+)") |
There was a problem hiding this comment.
- uses steps before approving pins
The pinning gate misses the common GitHub Actions shorthand form used elsewhere in this repo, such as - uses: actions/checkout@..., because this regex only matches lines where uses: appears immediately after indentation. An unpinned shorthand step like - uses: actions/checkout@v4 would therefore pass the new workflow-hardening check, leaving a false pass for the SHA-pinning policy the gate is intended to enforce.
Useful? React with 👍 / 👎.
| def _check_permissions_declared(path: Path, text: str, errors: list[str]) -> None: | ||
| # Minimal parser: require an explicit top-level permissions key before jobs. | ||
| before_jobs = text.split("\njobs:", 1)[0] | ||
| if "\npermissions:" not in f"\n{before_jobs}": | ||
| errors.append(f"{path}: missing explicit top-level permissions block") |
There was a problem hiding this comment.
G1 says top-level permissions must be {} or contents: read, but this check only verifies that the key appears before jobs. In a workflow with permissions: write-all or permissions: contents: write, the new hardening gate would pass even though the token is broader than the quality contract permits, so workflow PRs can regress least privilege without this gate catching it.
Useful? React with 👍 / 👎.
| def _line_is_allowed(line: str) -> bool: | ||
| lowered = line.lower() | ||
| return any(marker in lowered for marker in ALLOW_MARKERS) |
There was a problem hiding this comment.
Because _line_is_allowed accepts any allow marker anywhere on the line before testing risk patterns, a positive sentence such as LiouScope is production-ready and not merely a prototype would be skipped solely due to not . That creates a false pass for exactly the unsupported status claims this new gate is meant to block unless the marker is tied to the claim.
Useful? React with 👍 / 👎.
| if _line_is_allowed(line): | ||
| continue | ||
| for pattern in RISK_PATTERNS: | ||
| if pattern.search(line): |
There was a problem hiding this comment.
When a release status is actually proven, a line such as PyPI-published: https://pypi.org/... still reaches pattern.search(line) and is reported because the only bypass is _line_is_allowed, which contains negative/uncertainty markers rather than evidence markers. That contradicts the workflow's stated fix of adding evidence or linking a release audit, so the gate will block the evidence-lock docs it is supposed to permit.
Useful? React with 👍 / 👎.
| re.compile(r"\bDOI\b.*\b(complete|published|archived|released)\b", re.IGNORECASE), | ||
| re.compile(r"\bZenodo\b.*\b(complete|published|archived|released)\b", re.IGNORECASE), |
There was a problem hiding this comment.
These DOI/Zenodo patterns only match when DOI or Zenodo appears before words like published or archived, so common claims such as published DOI minted or published on Zenodo pass without any evidence marker. Since G4 is specifically meant to prevent unsupported DOI/Zenodo archival-status drift, the matcher needs to cover both word orders rather than only the noun-first phrasing.
Useful? React with 👍 / 👎.
| import sys | ||
| from pathlib import Path | ||
|
|
||
| ROOT = Path(__file__).resolve().parents[2] |
There was a problem hiding this comment.
The claim gate is advertised as protecting documentation/release/status wording, but it only scans README.md, CHANGELOG.md, and docs/. Public root Markdown such as SECURITY.md, CONTRIBUTING.md, or future GitHub-facing templates can still introduce unsupported production/PyPI/DOI claims without this workflow seeing them, leaving a gap in the drift protection.
Useful? React with 👍 / 👎.
1 participant
Summary
Adds a quality contract for LiouScope plus a fast CI gate that checks:
Why
The repo already has strong baselines: SHA-pinned actions, minimal permissions, CI matrix, Scorecard, zizmor, and guarded PyPI publish. The remaining gap is preventing documentation/release/status drift from entering the repo after CI is green.
Files
Quality / security notes
Verification
007 / provenance
This PR implements the follow-up from the 007 GitHub Quality Workflow OS round. It is deliberately a small, reversible quality gate rather than a large workflow rewrite.