Skip to content

[Feature] Incremental scanning, and the auto-resolution fix it requires #17

Description

@addyCooks

Summary

Enhancement 2 of #1: skip files unchanged since the last successful scan, instead of re-auditing a whole repository every run. #14 landed enhancement 1 (sentinel estimate plus the instrumentation) and deliberately left this half out, because it touches versioned contracts and needs sign-off first.

Splitting it out so the design can be agreed before code. Carrying forward the plan from #14 and the review agreement on it.

The blocker: incremental scanning breaks auto-resolution

This has to land in the same PR as the cache, not after it.

planReconciliation sees only this run's findings and the currently open issues. It cannot distinguish "the finding is gone" from "the file was never scanned" — any hash absent from this run bumps the miss counter, and at resolveAfterMisses (default 3) the issue is closed. source/dedup/plan.ts:116-125:

for (const [hash, issue] of openByHash) {
  if (findingByHash.has(hash)) continue;
  const misses = readMisses(issue.body) + 1;
  if (misses >= resolveAfterMisses) plan.toResolve.push(issue);
}

So on a daily schedule, skipping unchanged files silently closes real, unfixed findings after three runs. For a triage tool that is the worst possible failure: the tool quietly tells you a vulnerability is fixed because it stopped looking.

Fix: pass the scanned scope into the planner so an out-of-scope issue is held — neither refreshed nor missed — rather than aged out.

Constraint worth settling first

Holding an issue requires knowing which file it refers to, and the planner currently cannot tell.

  • ExistingIssue carries number, url, state, labels, body (source/issues/types.ts:36-42) — no path.
  • Issue bodies carry three markers: hash, misses, last-seen. The file appears only as prose in the title and a **Location:** line (source/issues/body.ts:50).
  • findingHash is sha256(rule + file + category) truncated to 16 chars (source/dedup/hash.ts:15-18) — opaque, so scope membership cannot be recovered from it.

So this needs a structured path marker written into issue bodies alongside hash, plus a migration answer for issues filed before it exists. Suggestion: treat an issue with no path marker as always in scope, so pre-existing issues keep today's behaviour and age out normally rather than being held forever. That keeps the change safe for existing installs, and the marker backfills naturally as issues are touched.

Parsing the path out of the **Location:** prose instead would work without a migration but is fragile, and would break the moment the body template changes.

Design

Agreed in review on #14:

  • Per-repo config. incremental: true|false per target, off by default until it is proven somewhere real.
  • Pack invalidation. Cache {name, version, body-hash} next to the scanned SHA. Any of the three changing forces a full pass for that pack — a pack whose prompt changed has to re-see everything, even unchanged files.
  • Forcing a full audit. Always available: a --full flag, plus automatic fallback to a full scan whenever the cached SHA is unreachable (shallow clone, force-push, first run).
  • Cache location. Committed state in the config repo, same posture as run records. No database.

Open questions

  • Cache granularity: per repo+pack, or per repo+pack+file? Per-pack is simpler and enough for the invalidation rule above.
  • depends_on: if a dependency pack's body-hash changes, does the dependent pack also need a full pass? Probably yes, since its prompt includes the dependency.
  • Does the held-issue state need surfacing in the run record and dashboard, so an operator can see "held, not scanned" separately from "still open"?

Acceptance

  • Skipping unchanged files cannot resolve an issue that was simply not scanned — with a test that runs the reconciliation across several incremental runs and asserts the issue stays open.
  • A pack change forces a full pass for that pack.
  • --full and the unreachable-SHA fallback both produce a complete scan.
  • Existing installs that do not opt in see byte-identical behaviour.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions