Skip to content

Add package filesystem evidence scanner#2189

Draft
jesse-merhi wants to merge 1 commit into
mainfrom
jesse/filesystem-evidence-scanner
Draft

Add package filesystem evidence scanner#2189
jesse-merhi wants to merge 1 commit into
mainfrom
jesse/filesystem-evidence-scanner

Conversation

@jesse-merhi
Copy link
Copy Markdown
Member

Summary

Adds a narrow pure scanner for package file contents that reports deterministic evidence for:

  • raw Node filesystem module imports and calls
  • OpenClaw fs-safe module imports and helper calls
  • bounded evidence buckets sorted by file and line

This intentionally does not add DB tables, background jobs, HTTP endpoints, admin UI, or CLI commands. Those can layer on top of this scanner in follow-up PRs.

Validation

  • bun run test convex/lib/packageFilesystemEvidenceScan.test.ts
  • bun run format:check
  • bun run lint
  • bunx tsc --noEmit --pretty false
  • git diff --cached --check

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented May 13, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
clawhub Ready Ready Preview, Comment May 13, 2026 1:03am

@Patrick-Erichsen
Copy link
Copy Markdown
Contributor

Nice, so this is to ensure plugin authors are using things like fs-safe correctly?

@github-actions
Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please update it or it will be closed.

@github-actions github-actions Bot added the stale label May 20, 2026
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P2 Normal backlog priority with limited blast radius. labels May 22, 2026
@clawsweeper
Copy link
Copy Markdown

clawsweeper Bot commented May 22, 2026

Codex review: found issues before merge. Reviewed May 24, 2026, 4:43 PM ET / 20:43 UTC.

Summary
Adds a Convex package filesystem evidence scanner and tests for raw Node fs usage, OpenClaw fs-safe usage, and bounded sorted evidence buckets.

Reproducibility: yes. for the review findings. GitHub CI reproduces the dead-code failure, and source inspection shows the pre-bound fs namespace plus line-oriented import matching that causes scanner false positives and false negatives.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🌊 off-meta tidepool
Patch quality: 🦐 gold shrimp
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Connect the scanner to an approved production/admin scan path or approved standalone entry so ci:static passes.
  • Add regressions for virtual fs objects and multi-line raw fs imports.
  • Update the package-scan spec if dedicated filesystem evidence is now intentionally in scope.

Risk before merge

  • The branch currently fails the required static gate because the new Convex production library has no production caller or approved standalone entry point.
  • If integrated as-is, the scanner can misclassify package-local virtual fs objects as raw Node filesystem usage and miss common multi-line raw fs imports, producing noisy or incomplete security evidence.
  • Maintainers still need to decide where dedicated filesystem evidence belongs because the current package-scan intent note keeps package static/LLM scan inputs metadata-only for now.

Maintainer options:

  1. Choose the integration path first (recommended)
    Approve the production/admin scan path or an explicit standalone entry before keeping this as production scanner code.
  2. Tighten binding detection
    Start raw fs namespaces empty, add bindings only from confirmed Node fs imports/requires, support multi-line imports, and cover virtual-fs plus multi-line regressions.
  3. Keep the draft paused
    Leave the PR unmerged until maintainers decide whether filesystem package evidence is in scope for the package security model.

Next step before merge
Maintainers need to choose the package evidence integration path and scope before automation can safely repair the branch.

Security
Cleared: The diff adds a pure text scanner and tests with no new dependency, secret handling, network access, install hook, or code execution path; security-signal correctness is covered by review findings.

Review findings

  • [P2] Wire the scanner into an approved entry point — convex/lib/packageFilesystemEvidenceScan.ts:78-85
  • [P2] Require a real fs binding before flagging calls — convex/lib/packageFilesystemEvidenceScan.ts:215
  • [P2] Handle multi-line raw fs imports — convex/lib/packageFilesystemEvidenceScan.ts:109-113
Review details

Best possible solution:

Land a binding-aware filesystem evidence scanner only through an approved package scan or admin entry point, with regression tests and an updated package-scan intent note.

Do we have a high-confidence way to reproduce the issue?

Yes for the review findings. GitHub CI reproduces the dead-code failure, and source inspection shows the pre-bound fs namespace plus line-oriented import matching that causes scanner false positives and false negatives.

Is this the best way to solve the issue?

No. The scanner is useful foundation work, but this branch should first choose an integration path and fix binding-aware detection before it becomes a maintainable package security signal.

Full review comments:

  • [P2] Wire the scanner into an approved entry point — convex/lib/packageFilesystemEvidenceScan.ts:78-85
    scanPackageFilesystemEvidence is exported from a new Convex lib file, but the PR intentionally adds no caller or approved standalone entry. The required ci:static job already fails in deadcode:files on this file, so this cannot merge as production code until it is integrated or explicitly exempted.
    Confidence: 0.93
  • [P2] Require a real fs binding before flagging calls — convex/lib/packageFilesystemEvidenceScan.ts:215
    rawFsNamespaces starts with fs, so package-local code such as const fs = makeVirtualFilesystem(); fs.readFile(path) would be reported as raw Node filesystem usage without any fs or node:fs import. Start empty and add a regression for virtual fs calls.
    Confidence: 0.9
  • [P2] Handle multi-line raw fs imports — convex/lib/packageFilesystemEvidenceScan.ts:109-113
    The scanner only binds raw fs helpers when the module literal and import syntax appear on the same physical line. Standard formatted imports split across lines will never add the helper or namespace binding, so later raw filesystem calls can escape the evidence scanner.
    Confidence: 0.84

Overall correctness: patch is incorrect
Overall confidence: 0.91

Codex review notes: model gpt-5.5, reasoning high; reviewed against 963b0a571943.

Label changes

Label justifications:

  • P2: This is a normal-priority package security scanner feature with limited blast radius, but it has blocking CI and correctness issues.
  • merge-risk: 🚨 security-boundary: The scanner is meant to produce package security evidence, and its current binding logic can misclassify filesystem usage if integrated.
  • merge-risk: 🚨 automation: The current diff breaks the required static dead-code gate by adding an unreferenced production Convex file.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: The contributor proof gate does not apply because this is a member-authored draft PR; the PR body lists command validation.
Evidence reviewed

What I checked:

  • Live PR state: Live GitHub state shows this PR is open, draft, member-authored, at head dedb88f, with the CI static check failing. (dedb88fa49c1)
  • Current main does not contain the scanner: Current main has static publish/package security helpers but no packageFilesystemEvidenceScan file or equivalent raw-fs/fs-safe evidence scanner. (963b0a571943)
  • Scanner is an isolated production helper: The PR exports scanPackageFilesystemEvidence from a new Convex lib file, but the diff adds no caller, DB field, HTTP/admin endpoint, CLI command, or approved standalone entry point. (convex/lib/packageFilesystemEvidenceScan.ts:78, dedb88fa49c1)
  • Static gate failure: The static CI job fails in deadcode:files because Knip reports convex/lib/packageFilesystemEvidenceScan.ts as unused. (dedb88fa49c1)
  • Dead-code policy covers Convex production files: Knip treats convex/**/*.ts as production project files while ignoring tests unless KNIP_INCLUDE_TESTS is set, so a test-only production helper fails the static gate. (knip.config.ts:61, 963b0a571943)
  • False-positive raw fs binding: createScanState pre-seeds rawFsNamespaces with "fs", so package-local fs objects can be flagged as raw Node filesystem calls without a confirmed node:fs binding. (convex/lib/packageFilesystemEvidenceScan.ts:215, dedb88fa49c1)

Likely related people:

  • Patrick-Erichsen: Current-main blame ties the static publish scan wrapper, package scan intent note, moderation engine scan loop, and package scan-state resolver to the v0.17.0 release commit authored by Patrick Erichsen. (role: recent package scanner/security contributor; confidence: high; commits: b753b1f7ab0e, 39107900ea2a; files: convex/lib/staticPublishScan.ts, convex/lib/moderationEngine.ts, convex/lib/packageSecurity.ts)
  • jesse-merhi: Beyond authoring this PR, Jesse has prior merged history on the package trust endpoint and package security surface that this scanner would feed into. (role: adjacent package trust contributor; confidence: medium; commits: f0a6789c31d5; files: convex/httpApiV1/packagesV1.ts, convex/lib/packageSecurity.ts, convex/packages.ts)
  • Deepak Jain: Recent commits by Deepak adjusted exposed-secret, environment-variable, and static moderation behavior that any package filesystem evidence scanner should align with or intentionally diverge from. (role: adjacent static moderation contributor; confidence: medium; commits: f20dd624a526, d855d09ab070, 932155cb8ff5; files: convex/lib/moderationEngine.ts, convex/lib/moderationEngine.test.ts, convex/lib/moderationReasonCodes.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper
Copy link
Copy Markdown

clawsweeper Bot commented May 22, 2026

ClawSweeper PR egg

🔥 Warming up: real-behavior proof passed; findings, security review, or rank-up moves are still in progress.

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

  • Merged PRs are hatchable.
  • Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
  • Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.
What is this egg doing here?
  • Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
  • The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
  • Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
  • The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
  • Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

@github-actions github-actions Bot removed the stale label May 23, 2026
@clawsweeper clawsweeper Bot added merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. labels May 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal backlog priority with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants