Skip to content

Pin the scanner's remaining evasion classes as expected-to-evade fixtures (successor to #7) #13

Description

@OriNachum

scripts/colleague_inventory.py now pins per-module spawn-site shapes, which
closes the architectural finding of #7: a new spawn in an already-allow-listed
module fails --check. Verified adversarially — appending one benign
subprocess.run(["true"]) to tools.py in a pinned clone produces
tools.py: baseline 2 site(s), observed 3 (+subprocess.run×1) and exit 1.

What remains open, and is documented in the module docstring rather than
silently dropped:

  • Indirectionsp = subprocess, runner = subprocess.run,
    getattr(subprocess, "run"), importlib.import_module, a dict of callables,
    functools.partial, a class attribute. Only import statements bind;
    closing this needs assignment-alias dataflow.
  • Structure — sibling re-export inside colleague/ (from .shim import run), exec() of a source string, a spawn in a file outside the package,
    a symlinked subdirectory rglob will not follow.
  • shell=True spelling — detection still requires a literal
    ast.Constant, so shell=SH and **{"shell": True} report nothing. This
    now matters slightly more, because an argv-to-shell upgrade spelled
    non-literally will not change a site's signature.
  • Two deliberate trades from the scope-aware rewrite — rebinding
    subprocess inside a function hides every spawn in it, and removing one
    spawn while adding another of the same shape in the same module is invisible.

The proposal

The task that closed the architectural hole deliberately did not add
regression tests for these, and gave a good reason: adding them as ordinary
tests would imply they are closed. The honest form is to commit them as
expected-to-evade fixtures — tests that assert the scanner currently
misses them, so the known hole is pinned rather than papered over, and any
future fix flips a test from expected-evasion to caught.

That also gives #7's remaining acceptance criterion ("the evasion fixtures from
this report become regression tests") a way to be satisfied honestly.

Fixture safety boundary, already established and enforced by
test_fixture_sources_stay_benign: every committed fixture argv stays benign
(true, echo), none of #7's illustrative destructive payloads appear, and no
fixture requires network. Fixtures are source text for ast.parse and never
execute.

The honest ceiling stays what it has been: detects accidental and careless
drift from a pinned baseline
. A static AST scanner cannot stop a determined
author.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions