Merge pull request #316 from workglow-dev/claude/sharp-lamport-ata6g5-proxy-approval-gate

fix(spac): require approval evidence before a general definitive proxy emits a proxy event

Author: sroussey

CLAUDE.md

@@ -1695,7 +1695,7 @@ which no deal walk reads):
 | `2.01` | `completed`            | unconditional                                                                                        |
 | `1.01` | `definitive_agreement` | the submission carries a merger-shaped EX-2 exhibit AND the event is not dated before the date floor |
 | `1.02` | `terminated`           | a deal was pending as of that filing, or the exhibits are merger-shaped                              |
-| `5.07` | `vote`                 | the pending deal had a definitive-agreement or proxy date                                            |
+| `5.07` | `vote`                 | a deal was pending as of that filing AND it had a proxy date                                         |
 
 Item `8.01` carries **no lifecycle mapping at all** — `mapItemCodesToSpacEvents`
 falls through its `default: break` and writes nothing. A termination disclosed
@@ -1842,15 +1842,123 @@ consideration) and observes the target company (`relation: "merger-proxy:target"
 correlates each extraction onto the matching `spac_deal` by filing-date window —
 _deriving_ `target_name` / `target_cik` / `pipe_amount` (a later filing supersedes
 an earlier one — definitive over preliminary, revised over definitive), which
-retires the 8-K path's positional merge-preserve. Only the **definitive merger**
-statements `DEFM14A` and `DEFM14C` emit the `proxy` event (→ `proxy_date` /
-`status = proxy`): a consent deal (14C) has no `8-K 5.07` vote, so the definitive
-14C is its only approval-stage signal. Preliminary (`PREM14A`/`PREM14C`) and revised
-(`DEFR14A`/`PRER14A`) proxies are extraction-only. S-4 is deferred (newco-CIK linkage). Configure the
+retires the 8-K path's positional merge-preserve. Preliminary (`PREM14A`/`PREM14C`)
+and revised (`DEFR14A`/`PRER14A`) proxies are extraction-only. S-4 is deferred
+(newco-CIK linkage). Configure the
 model via `SEC_MERGER_PROXY_MODEL` (default `claude-sonnet-5`) and an optional
 confidence floor via `SEC_MERGER_PROXY_CONFIDENCE_FLOOR` (falls back to the shared
 `SEC_S1_CONFIDENCE_FLOOR` when unset).
 
+#### Which statements emit the `proxy` event
+
+The `proxy` event (→ `proxy_date` / `status = proxy`) is **two-tier**, and the
+tiers differ in what counts as evidence:
+
+- The **definitive merger** statements `DEFM14A` / `DEFM14C` emit it on the form
+  symbol alone — the symbol says the meeting is about a combination, so the
+  event still lands when the merger section is absent or low-confidence and the
+  section dead-letters. (A consent deal (14C) has no `8-K 5.07` vote, so the
+  definitive 14C is its only approval-stage signal.)
+- The **general definitive** statements `DEF 14A` / `DEF 14C` — which is where
+  most SPACs actually vote their combination — emit it only on **two**
+  conjunctive pieces of document evidence: an extracted deal AND
+  `seeksCombinationApproval` (`proxies-information-statements/seeksCombinationApproval.ts`),
+  a deterministic scan for a numbered proposal item naming the filer's defined
+  `Business Combination Proposal`, or a request to approve/adopt the **agreement**
+  (business combination agreement / agreement and plan of merger / merger
+  agreement).
+
+The extracted deal alone is not evidence, because an **extension** proxy recites
+the announced combination at length: `S1_SECTIONS.BUSINESS_COMBINATION` accepts a
+bare `The Business Combination` heading, so the section is found, the model
+returns a target, and the filing looked exactly like a merger proxy. That is a
+silent corruption, not a missing row — a `proxy` event OPENS a deal by itself in
+`spacDealGrouping.ts`, which makes the vehicle's next item `5.07` a merger
+`vote`, which makes any Form 25/15 inside the 90-day post-approval window a
+`completed` de-SPAC, with `surviving_name` promoted onto `current_name`.
+
+The gate is deliberately **deterministic rather than a model schema field**.
+The failure costs are asymmetric: a false positive corrupts the primary answer
+with no trace, while a false negative only degrades a fallback — a real close
+files an Item 2.01, which maps to `completed` unconditionally. A model field
+also could not repair existing rows without re-paying the AI bill, and making it
+required would force an extractor version cycle. Same reasoning as the ownership
+subtotal and risk-caption heading guards: enforce it, don't trust the prompt.
+
+Two rules the patterns must keep, both measured over 348 real SIC-6770
+`DEF 14A` / `DEF 14C` statements (9 merger proxies, 339 extension / annual /
+other; the rule scores 9/9 recall at **0 false positives**):
+
+- **Never match the term anywhere in the document.** Every extension proxy
+  carries "as if they had voted against a business combination proposal", and
+  many cross-reference the combination's own proposal in another filing. A
+  whole-document test for `business combination proposal` fired on 24 of the
+  348, **all** of them extension or annual meetings. Both patterns are therefore
+  line-shaped (≤ 300 chars) and the defined-term one is anchored at line start.
+- **The approval object must be the AGREEMENT.** A bare
+  `to approve … business combination` is the standard extension wording — "to
+  approve an amendment … to extend the date by which the Company must consummate
+  a business combination".
+
+There is deliberately no extension-exclusion term: a proxy asking for an
+extension AND for approval of the combination is a genuine merger proxy and must
+still emit.
+
+Extraction is unchanged either way — the `spac_merger_extraction` row is written
+whether or not the gate passes — and the verdict is recorded on it as
+`seeks_combination_approval`. `NULL` means the gate was not evaluated: the row
+predates it, or the form symbol alone decides (the "M" forms never pay the
+full-document render). The backfill keys on that NULL, so it must stay
+distinguishable from a recorded `false`.
+
+**Retraction.** `recordMergerProxy` deletes a `proxy` event for the accession
+when the caller now decides the filing is not approval-stage, mirroring the
+sibling deletes in `recordDeregistration` / `recordUnitSplit` / `recordCompleted`.
+Reclassification therefore runs in both directions and a replay demotes the deal
+instead of leaving the old verdict standing. The delete is scoped to that one
+accession, so it can only retract what a previous run of the same filing wrote.
+
+No extractor version bump: the persisted extraction rows are unchanged and still
+correct, and the derived event is rebuildable from the document with no model
+call.
+
+**Recovery ceremony.** Databases populated before the gate existed carry false
+closes. Re-run the proxies, then let the listing-removal classifier re-derive
+the Form 25/15 verdicts that were built on them:
+
+```bash
+sec extractor backfill merger-proxy   # re-derives the verdict; retracts stale proxy events
+sec extractor backfill 25-15          # repeat until it reports `processed 0`
+```
+
+The first re-selects exactly the general definitive proxies whose
+`seeks_combination_approval` is still NULL, and extinguishes itself once each
+has a verdict. The second is the fixpoint sweep documented above: `filterTodo`
+re-derives each accession's kind against the live classifier, so a stale
+`completed` on an earlier accession keeps a later one classifying `completed`
+until the earlier one is corrected.
+
+Find the affected rows first — every `proxy` event on a general definitive form,
+and the closes standing on one:
+
+```sql
+SELECT cik, accession_number, event_date, form FROM spac_event
+WHERE event_type = 'proxy' AND form IN ('DEF 14A','DEF 14C') ORDER BY cik, event_date;
+
+SELECT p.cik, c.accession_number AS close_accession, c.form, c.event_date
+FROM spac_event p JOIN spac_event c ON c.cik = p.cik AND c.event_type = 'completed'
+WHERE p.event_type = 'proxy' AND p.form IN ('DEF 14A','DEF 14C')
+  AND c.form IN ('25','25/A','25-NSE','25-NSE/A','15-12B','15-12G','15-15D','20-F','20-F/A');
+```
+
+⚠️ Expect status **regressions** on real CIKs as the false closes unwind —
+`completed` back to `searching` / `deal_announced`, and `surviving_name` /
+`post_merger_*` / the `current_*` promotion dropping back to the `spac_*` mirror
+(those five columns are derived strictly from a completed deal and are never
+merged forward). That is the correction, not a loss: every change is captured in
+`spac_history` / `ChangeLog`, and genuinely completed SPACs are re-filled by
+`sec spac backfill-despac`.
+
 A proxy ingested before its issuer's `spac` row exists (e.g. the S-1 lands later)
 hits the known-SPAC gate and no-ops — recording a successful run, so the normal
 unprocessed-run sweep never revisits it. `sec spac backfill-merger-proxies`
@@ -2141,7 +2249,9 @@ by default over all filings of its forms; extractors whose candidate set is
 narrower add a descriptor entry (the `redemption` / `loi` sub-extractors select
 known-SPAC trigger-item 8-Ks) and extractors whose recorded success can be a
 gated no-op override `filterTodo` (`merger-proxy` keeps candidates lacking a
-`spac_merger_extraction` row, since its known-SPAC gate records `success: true`).
+`spac_merger_extraction` row, since its known-SPAC gate records `success: true`,
+plus the general definitive proxies whose `seeks_combination_approval` verdict is
+still NULL — a self-extinguishing clause, since the re-run records one).
 The default needing-work predicate is a bulk anti-join against `extractor_runs`
 at the active version, exported as `defaultFilterTodo` so a descriptor that only
 WIDENS it does not restate it. The `redemption` / `loi` descriptors do exactly

src/sec/forms/proxies-information-statements/Form_DEFM14A.storage.e2e.test.ts

@@ -159,11 +159,45 @@ describe("processMergerProxy (e2e)", () => {
     expect(row?.target_name).toBe("Acme Target Inc.");
   });
 
+  /** The merger prose the stub model's source_span must verify against. */
+  const MERGER_PROSE =
+    "<h1>The Business Combination</h1>\n" +
+    "<p>\n" +
+    "Merge SPAC Inc., a blank check company, has entered into a business combination\n" +
+    "with Acme Target Inc., a leading operating company in its sector. Upon the closing\n" +
+    "of the business combination with Acme Target Inc., the post-combination company\n" +
+    "will continue the business of Acme Target Inc.\n" +
+    "</p>";
+
+  /**
+   * A real DEF 14A combination vote: the notice enumerates the combination as a
+   * ballot item, and the section the extractor reads sits below it. The two are
+   * separate lines in a real filing — the segmenter's `The Business
+   * Combination` heading is a section title, not a proposal — so the approval
+   * evidence and the extracted deal genuinely come from different places.
+   */
+  const APPROVAL_BODY =
+    "<h1>Proposal No. 1 — The Business Combination Proposal</h1>\n" +
+    "<p>To approve and adopt the Business Combination Agreement, dated as of March 1, 2021.</p>\n" +
+    MERGER_PROSE;
+
+  /**
+   * An extension proxy that RECITES the announced combination: the segmenter
+   * finds the same business-combination section and the extractor returns the
+   * same deal, but the only thing on this ballot is the charter extension.
+   */
+  const EXTENSION_BODY =
+    MERGER_PROSE +
+    "\n<h1>Extension Amendment Proposal</h1>\n" +
+    "<p>To amend the Company's charter to extend the date by which it must consummate\n" +
+    "an initial business combination from May 1, 2021 to November 1, 2021.</p>";
+
   it("emits a proxy event for a DEF 14A whose merger section yields a deal", async () => {
     // Most SPACs vote their combination on a plain DEF 14A, never a DEFM14A.
+    // The heading is the ballot item, so the approval gate passes too.
     await seedSpacWithOpenDeal(120);
     cleanup = scriptMergerDeal();
-    await runProxy(120, "120-def14a", "DEF 14A");
+    await runProxy(120, "120-def14a", "DEF 14A", "2021-05-01", submissionWithBody(APPROVAL_BODY));
 
     const events = await repo.getEvents(120);
     expect(events.filter((e) => e.event_type === "proxy")).toHaveLength(1);
@@ -173,6 +207,64 @@ describe("processMergerProxy (e2e)", () => {
     expect(row?.target_name).toBe("Acme Target Inc.");
   });
 
+  it("emits no proxy event for an extension DEF 14A that describes the announced combination", async () => {
+    // The failure this gate exists to prevent. The merger section is present
+    // and yields a deal, so the extracted-deal test alone passes — but the
+    // meeting approves only the extension. A proxy event here opens a deal on
+    // its own, which turns the next item 5.07 into a merger vote and makes any
+    // Form 25/15 inside 90 days read as a completed de-SPAC.
+    await seedSpacWithOpenDeal(124);
+    cleanup = scriptMergerDeal();
+    await runProxy(
+      124,
+      "124-def14a-ext",
+      "DEF 14A",
+      "2021-05-01",
+      submissionWithBody(EXTENSION_BODY)
+    );
+
+    const events = await repo.getEvents(124);
+    expect(events.some((e) => e.event_type === "proxy")).toBe(false);
+    // Extraction is unchanged: the row is still written, with the verdict.
+    const extraction = await new SpacMergerExtractionRepo().getByAccession("124-def14a-ext");
+    expect(extraction?.target_name).toBe("Acme Target Inc.");
+    expect(extraction?.seeks_combination_approval).toBe(false);
+
+    const row = await repo.getSpac(124);
+    expect(row?.status).toBe("deal_announced");
+    expect(row?.proxy_date).toBeNull();
+  });
+
+  it("a retracted proxy event demotes the deal on replay", async () => {
+    // Reclassification runs in both directions: reprocessing the same
+    // accession with a body the gate rejects must remove the event the earlier
+    // run wrote, not leave the stale approval stage standing.
+    await seedSpacWithOpenDeal(125);
+    cleanup = scriptMergerDeal();
+    await runProxy(125, "125-def14a", "DEF 14A", "2021-05-01", submissionWithBody(APPROVAL_BODY));
+    expect((await repo.getSpac(125))?.status).toBe("proxy");
+
+    await runProxy(125, "125-def14a", "DEF 14A", "2021-05-01", submissionWithBody(EXTENSION_BODY));
+
+    const events = await repo.getEvents(125);
+    expect(events.some((e) => e.event_type === "proxy")).toBe(false);
+    const row = await repo.getSpac(125);
+    expect(row?.proxy_date).toBeNull();
+    expect(row?.status).toBe("deal_announced");
+  });
+
+  it("records the gate verdict only for the forms it governs", async () => {
+    // The "M" forms decide on the symbol alone and must not pay a full-document
+    // render, so their verdict stays null — which is also what the backfill
+    // keys on to re-select the general definitive proxies that need one.
+    await seedSpacWithOpenDeal(126);
+    cleanup = scriptMergerDeal();
+    await runProxy(126, "126-defm", "DEFM14A");
+    expect(
+      (await new SpacMergerExtractionRepo().getByAccession("126-defm"))?.seeks_combination_approval
+    ).toBeNull();
+  });
+
   it("emits no proxy event for a DEF 14A extension proxy with no merger section", async () => {
     // The form symbol is not evidence: an extension vote is the common DEF 14A.
     // Its proposal heading matches none of the merger section patterns, so no
@@ -225,7 +317,7 @@ describe("processMergerProxy (e2e)", () => {
   it("a DEF 14A proxy event makes the following item 5.07 a merger vote", async () => {
     await seedSpacWithOpenDeal(123);
     cleanup = scriptMergerDeal();
-    await runProxy(123, "123-def14a", "DEF 14A");
+    await runProxy(123, "123-def14a", "DEF 14A", "2021-05-01", submissionWithBody(APPROVAL_BODY));
 
     const form8K = await Form_8_K.parse("8-K", "<html/>");
     await processForm8K({

src/sec/forms/proxies-information-statements/Form_DEFM14A.storage.ts

@@ -4,7 +4,12 @@
  * SPDX-License-Identifier: Apache-2.0
  */
 
-import { globalServiceRegistry, type IExecuteContext, type ModelConfig } from "workglow";
+import {
+  globalServiceRegistry,
+  renderMarkdown,
+  type IExecuteContext,
+  type ModelConfig,
+} from "workglow";
 import { prefetchModel } from "../../../task/model/EnsureModelDownloadedTask";
 import { buildEntityObserver } from "../../../resolver/buildEntityObserver";
 import { CanonicalCompanyRepo } from "../../../storage/canonical/CanonicalCompanyRepo";
@@ -34,9 +39,11 @@ import {
 } from "../registration-statements/s1/mergerModel";
 import type { FormS1Parsed } from "../registration-statements/Form_S_1";
 import {
+  GENERAL_DEFINITIVE_PROXY_FORMS,
   MERGER_PROXY_OPTIONAL_FORMS,
   MERGER_PROXY_SECTION,
 } from "../../../storage/versioning/extractorIds";
+import { seeksCombinationApproval } from "./seeksCombinationApproval";
 
 const EXTRACTOR_ID = "merger-proxy";
 // Stays 1.0.0: no persisted data to re-extract, so the target_description
@@ -50,22 +57,6 @@ const MERGER_SECTION = MERGER_PROXY_SECTION;
  */
 const DEFINITIVE_PROXY_FORMS = new Set(["DEFM14A", "DEFM14C"]);
 
-/**
- * General definitive statements. Most SPACs never file a `DEFM14A` at all —
- * across SIC 6770 filers a plain `DEF 14A` reaches 575 distinct SPACs against
- * `DEFM14A`'s 234 — so refusing these dropped the proxy stage for the majority
- * of vehicles, and with it every downstream rule anchored on `proxy_date` (the
- * 5.07 vote mapping, the post-approval listing-removal window).
- *
- * The form symbol alone is NOT evidence here: most filings on these forms are
- * annual meetings and charter-extension votes. They emit the event only when
- * this filing actually IS a merger proxy, evidenced by an extracted deal.
- * Preliminary (`PRE*`), revised (`DEFR14A`/`PRER14A`) and supplemental
- * (`DEFA14A`) statements stay out either way: only a definitive statement is an
- * approval-stage signal.
- */
-const GENERAL_DEFINITIVE_PROXY_FORMS = new Set(["DEF 14A", "DEF 14C"]);
-
 export interface ProcessMergerProxyArgs {
   readonly cik: number;
   readonly file_number: string;
@@ -91,7 +82,7 @@ export interface ProcessMergerProxyArgs {
  * it on the form symbol alone, so it still advances `proxy_date` when the
  * merger section is absent or low-confidence and the section dead-letters; a
  * {@link GENERAL_DEFINITIVE_PROXY_FORMS} one emits it only when this run
- * extracted a deal.
+ * extracted a deal AND the document asks shareholders to approve it.
  */
 export async function processMergerProxy(args: ProcessMergerProxyArgs): Promise<void> {
   const { cik, accession_number, form, filing_date, formMergerProxy } = args;
@@ -154,10 +145,17 @@ export async function processMergerProxy(args: ProcessMergerProxyArgs): Promise<
 
   // Segment; PARSE_ERROR dead-letters the merger section so a retry can resolve it.
   let byName: Map<S1SectionName, string>;
+  // null = not evaluated. Rendering the whole document is worth paying for only
+  // on the general definitive forms, whose symbol does not say the meeting is
+  // about a combination; the "M" forms decide on the symbol alone.
+  let seeks_combination_approval: boolean | null = null;
   try {
     const doc = parseEdgarHtml(formMergerProxy.html, `${form} ${accession_number}`);
     const sections = new DocumentTreeSegmenter().segment(doc);
     byName = new Map<S1SectionName, string>(sections.map((s) => [s.name, s.text]));
+    if (GENERAL_DEFINITIVE_PROXY_FORMS.has(form)) {
+      seeks_combination_approval = seeksCombinationApproval(renderMarkdown(doc));
+    }
   } catch (err) {
     const message = err instanceof Error ? err.message : String(err);
     await deadLetters.record({
@@ -303,6 +301,7 @@ export async function processMergerProxy(args: ProcessMergerProxyArgs): Promise<
             merger_consideration: deal.merger_consideration,
             confidence: deal.confidence,
             source_span: boundSourceSpan(deal.source_span),
+            seeks_combination_approval,
             model_id,
             created_at: now,
           });
@@ -327,7 +326,9 @@ export async function processMergerProxy(args: ProcessMergerProxyArgs): Promise<
       primary_document: args.primary_doc ?? null,
       emitProxyEvent:
         DEFINITIVE_PROXY_FORMS.has(form) ||
-        (GENERAL_DEFINITIVE_PROXY_FORMS.has(form) && extractedDeal),
+        (GENERAL_DEFINITIVE_PROXY_FORMS.has(form) &&
+          extractedDeal &&
+          seeks_combination_approval === true),
     });
   } catch (err) {
     const message = err instanceof Error ? err.message : String(err);