fix(s1): a deterministic pass may not preempt what it cannot supply

[sroussey]

A deterministic pass declares what it can supply (covers) and is allowed to
replace the AI call only when that covers everything the section rewrites
(clears). Four sections declared both sets at TABLE granularity while their
parse fills only some of the table's columns, so preempts returned true
unconditionally — and the section's clear(accession_number) has already run by
then. The columns the parse cannot read were written as NULL, the section
resolved clean, no dead letter was recorded, and every subsequent replay took
the same path, so nothing self-corrected.

The four over-claiming sections, and exactly what each nulled

section	parse	columns rewritten as null
underwriters	parseSpacUnderwriters	underwriter_link.role_detail, underwriter_link.over_allotment_shares (both hardcoded null)
beneficial ownership	parseBeneficialOwnership	security_class, shares_offered, shares_after, percent_after, footnote — and is_selling_stockholder: false
management	parseManagementRoster	person_observation.bio (observePerson UPSERTS the row, so the bio is genuinely rewritten)
sponsor promote	parseSpacPromoteTerms	none hardcoded — the || gate returns a row on ONE of two anchors, so the other five columns are rewritten from a partial read

is_selling_stockholder: false is the worst of these: it is not an absent value
but a positive false claim that the holder registers no resale.

The failure is concrete. An S-1 processed before this branch has
underwriter_link.role_detail = "bookrunner"; sec extractor backfill S-1 --force clears the links, the deterministic pass wins, and the rows come back
with role_detail = NULL. Form_S_1.storage.offering.test.ts PINNED that
behaviour (expect(links[0]!.role_detail).toBeNull()), which is why the
assertion is inverted here.

The discipline already existed and was simply not applied — spac-profile,
related-party and offering-terms each declare a covers that is a strict
subset of their clears and correctly decline.

Why column granularity needs no change to preempts

Destination names are opaque strings compared by exact match. So
covers: {underwriter_link.shares_allocated} does not contain
underwriter_link.role_detail, and the pass declines — no new comparison logic,
no "bare name expands to all its columns" convenience (which would reopen the
hole in one word). Mixing granularity for one table — bare in one set, qualified
in the other — also declines, in both directions, so a half-migrated
declaration fails safe rather than silently claiming or silently losing a table.
Existing table-granularity pairs (spac-sponsors, spac-classification) are
untouched and keep working.

The rule is now stated on DeterministicPass: a table is named bare only
when the parse fills every column persist writes for it; the moment one column
is beyond the parse, the whole table is named column by column in BOTH sets.

Why covers became text-dependent for promote and ownership

Naming the columns alone turns all four passes off. Two are bought back, because
for them a static set cannot answer the actual question — whether a null column
is a loss or is what the filing says:

• ownership — a SPAC's pre-IPO table prints one class, no "Shares Offered"
  and no "after the offering" columns, no selling stockholders and no footnote
  markers. There null IS the disclosure. A resale registration's table prints
  all of them, and there the same nulls delete stated figures.
  ownershipCoverage(text) reads that off the same headers and rows the parse
  walks.
• sponsor promote — this is a recall bug wearing the other bug's clothes:
  nothing is hardcoded, walkFields attempts all seven columns, and the defect
  is the || gate that returns a row on one anchor.
  promoteCoverage(text) re-runs walkFields and claims a column only when
  this filing's tables state it, which makes the gate stop mattering.

covers is therefore widened to ReadonlySet<string> | ((text: string) => ReadonlySet<string>), resolved once before extract and wrapped in
try/catch returning the empty set — a coverage function reads the section, so
it can fail like any parse, and declining costs one model call where propagating
would abort a section the model could still have extracted. The enforced rule:
a coverage function must be derived from the SAME walk extract performs, or it
is a second implementation that can disagree with the parse it speaks for.

Underwriters and management stay off. Their gaps are parser features, not
declaration fixes, and are follow-up work:

• underwriter_link.role_detail — the role is prose beside the syndicate table
  ("sole book-running manager", "co-manager"); teaching the parser to read that
  sentence is what would let the column join covers.
• person_observation.bio — the biography is the paragraphs following the
  roster table, which the table walk never reads.

Management roster closure resolves as a side effect

complete: () => false on the management pass, combined with if (meta.complete)
in the persist, made closeUnassertedPersonRoles for s1:management dead for
every filing whose roster table parses — the normal case — while CLAUDE.md still
names s1:management as one of only two closing populations.

No fix to complete was needed: column-qualifying management stops it
preempting, so complete is computed on the model path exactly as it was before
this branch and closure runs again. complete: () => false is deleted — a
complete on a pass that never preempts is dead config that reads as a live
decision. The durable argument now sits on the clears/covers pair: the
roster TABLE is not the roster POPULATION — a director named only in the prose
below it is invisible to the walk — so even a zero-decline parse could not close
a role. Form_S_1.storage.management.test.ts gains
"closes a role dropped from an amended roster", which fails if any future pass
declaration disables closure again.

Decision on use_of_proceeds.note and executive_compensation.footnote

Both are hardcoded null by their parse while the model is asked for them, and
they are the two remaining judgement calls. No database is reachable from this
environment, so the fill rate could not be measured; the basis below is the
section prompts and the committed fixtures/labels, and it splits the two.

The question that separates them is not "does the model fill it" but "is the
field's content recoverable from the row the parse does supply":

• use_of_proceeds.note stays bare (the pass keeps preempting). The prompt
  directs every qualifier a line item carries into purpose — "the row label
  copied WHOLE, including any parenthetical the cell carries" — and the parse
  copies that same cell verbatim. So note holds nothing the row does not still
  say. Corroborating: all 42 golden-labelled use-of-proceeds filings label only
  purpose and amount, and the extractor's eval compareFields is exactly
  ["purpose", "amount"] — the field has never been verified by anything.
• executive_compensation.footnote is column-qualified and that pass goes
  off. Nothing redirects footnote text anywhere else: the prompt tells the model
  to STRIP footnote markers out of person_name and out of every money field,
  so whatever a footnote says about a row appears on that row in no other
  column. Nulling it is a real loss, and the fail-safe answer costs one model
  call.

⚠️ Acute exposure — this should land before anyone runs the re-key ceremony

The documented re-key ceremony's step 3a is a corpus-wide backfill:

for id in S-1 D C CFPORTAL 1-A 1-Z 3 4 5 144 424; do sec extractor backfill "$id"; done

Run today, on this branch, that would silently downgrade every SPAC's
underwriter roles and ownership columns to null — every one of those sections
resolving clean, with no dead letter and nothing in any coverage number to show
it happened.

Also note that rows already written by #308 do not self-correct. A filing
whose underwriters/ownership/management/promote section already took the
deterministic path keeps its nulls until it is re-extracted:
sec extractor backfill S-1 --force and sec extractor backfill 424 --force,
re-paying the AI cost for those sections.

Tests

• new s1/deterministicPass.test.ts — declines a column covers omits; still
  preempts a table-granularity pair; declines a mixed-granularity pair in both
  directions; resolves a function-valued covers against the section text;
  declines when the coverage function throws; declines an undefined clears.
• Form_S_1.storage.offering.test.ts — the underwriters test is inverted and
  renamed to "does not preempt the underwriters model on a table it cannot read a role from"; the promote fixture now states all seven figures and keeps its
  deterministic assertion, plus a new "falls through to the model when the promote table states no trust total".
• Form_S_1.storage.ownership.test.ts — split into the SPAC table (still
  deterministic under ownershipCoverage) and a resale table with Shares
  Offered / Shares After columns (model runs, security_class and shares_after
  stored).
• Form_S_1.storage.management.test.ts / .compensation.test.ts — inverted so a
  parseable table still calls its model, and bio / footnote are stored
  non-null.

Verification

$ bunx vitest run src/sec/forms/registration-statements
 Test Files  10 failed | 72 passed (82)
      Tests  10 failed | 750 passed (760)

$ bun run test
 Test Files  12 failed | 415 passed | 3 skipped (430)
      Tests  18 failed | 3818 passed | 20 skipped (3856)

$ bunx prettier --check .
[warn] src/index.ts

All 18 failures are Error: Test timed out — zero assertion failures
(grep -c "Test timed out in" == 18). They are the golden-corpus fixture
loaders (parse*.corpus.test.ts × 10, parseEdgarHtml.golden.test.ts) and the
sec version CLI subprocess tests, none of which touch this change — the run
happened on a shared box under load average ~22. Re-running four of them alone
with --testTimeout=300000 passes clean:

$ bunx vitest run --testTimeout=300000 \
    src/sec/forms/registration-statements/s1/parseRelatedPartyTables.corpus.test.ts \
    src/sec/forms/registration-statements/s1/parseBeneficialOwnership.corpus.test.ts \
    src/sec/forms/registration-statements/s1/parseOfferingTables.corpus.test.ts \
    src/sec/forms/registration-statements/s1/parseManagementRoster.corpus.test.ts
 Test Files  4 passed (4)
      Tests  13 passed (13)

src/index.ts is the only prettier failure and is the pre-existing one a
sibling PR is fixing; it is not touched here. npx tsc --noEmit exits 0.

Also in this PR: classifier_source's schema description gains | deterministic,
and the eleven source?: "deterministic" JSDoc lines are corrected — "Persist-only;
never part of the model JSON schema" was wrong about persist, which reads
SectionPersistMeta.source and never this field. The field itself stays: it marks
a row as parser-produced and is asserted by the parser unit tests.

---

🤖 Generated with Claude Code

https://claude.ai/code/session_01LowBJQsCghLDiHwPN6FgUT

---

Generated by Claude Code