From 90cf52675b0606fafea0ae52faf67d4be799acb7 Mon Sep 17 00:00:00 2001 From: Alexey Orlenko's AI Agent Date: Mon, 1 Jun 2026 18:25:39 +0200 Subject: [PATCH] docs(sql-orm-many-to-many): add project spec and plan (TML-2597) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Shape the end-to-end many-to-many project: spec + 4-slice plan (0 contract+resolver foundation, gating parallel read/filter/write). Promoted from TML-2597 to a dedicated Linear Project; slices TML-2784..2787. No code changes — planning artifacts only. Signed-off-by: Alexey Orlenko's AI Agent --- projects/sql-orm-many-to-many/README.md | 7 ++ projects/sql-orm-many-to-many/plan.md | 47 ++++++++++ projects/sql-orm-many-to-many/spec.md | 105 ++++++++++++++++++++++ projects/sql-orm-many-to-many/trace.jsonl | 5 ++ 4 files changed, 164 insertions(+) create mode 100644 projects/sql-orm-many-to-many/README.md create mode 100644 projects/sql-orm-many-to-many/plan.md create mode 100644 projects/sql-orm-many-to-many/spec.md create mode 100644 projects/sql-orm-many-to-many/trace.jsonl diff --git a/projects/sql-orm-many-to-many/README.md b/projects/sql-orm-many-to-many/README.md new file mode 100644 index 0000000000..1f6d7ffcd5 --- /dev/null +++ b/projects/sql-orm-many-to-many/README.md @@ -0,0 +1,7 @@ +# SQL ORM — Many-to-Many End to End + +Transient project workspace for [TML-2597](https://linear.app/prisma-company/issue/TML-2597/sql-orm-complete-end-to-end-many-to-many-support-include-path-nested). See [`spec.md`](./spec.md) for the project spec and [`plan.md`](./plan.md) for the slice sequencing. + +Branch: `tml-2597-sql-orm-complete-end-to-end-many-to-many-support-include`. + +> Everything under `projects/` is transient — migrated to `docs/` or deleted at close-out per [`projects/README.md`](../README.md). diff --git a/projects/sql-orm-many-to-many/plan.md b/projects/sql-orm-many-to-many/plan.md new file mode 100644 index 0000000000..6071bbd9d1 --- /dev/null +++ b/projects/sql-orm-many-to-many/plan.md @@ -0,0 +1,47 @@ +# SQL ORM — Many-to-Many End to End — Plan + +**Spec:** `projects/sql-orm-many-to-many/spec.md` +**Linear Project:** [SQL ORM: Many-to-Many End to End](https://linear.app/prisma-company/project/sql-orm-many-to-many-end-to-end-c178df40ca3a) (planning record: TML-2597, `Plan: …`, Done) + +## At a glance + +Four slices: one **foundation slice** (slice 0) that makes M:N a validatable contract shape and surfaces the shared `through` descriptor, then a **three-way parallel fan-out** — read, filter, write — each consuming slice 0's hand-off and independent of the others. + +## Composition + +### Stack (deliver in order) + +1. **Slice `00-contract-resolver-foundation`** — Linear: [TML-2784](https://linear.app/prisma-company/issue/TML-2784) + - **Outcome:** An M:N relation (`rel.manyToMany` with `through`) emits a contract that round-trips `validateContract`; the shared resolver surfaces a uniform `through` descriptor (and the junction's required-non-FK-column info); the cardinality tag is canonicalised on `'N:M'` repo-wide. + - **Builds on:** None (correlated-only read path from TML-2729 / TML-2657 already on `main`). + - **Hands to:** (a) a validatable M:N contract shape — `through: { table, parentColumns, childColumns }` declared in the JSON schema + arktype validator + `ContractReferenceRelation` type; (b) `ResolvedRelation.through` + required-payload-column info on `resolveModelRelations`; (c) a single `'N:M'` cardinality tag with no `'M:N'` left in sql-orm-client. + - **Focus:** contract surface (`packages/2-sql/1-core/contract` validator, `data-contract-sql-v1.json`, `ContractReferenceRelation` type, delete the `as ContractRelation['cardinality']` cast in `build-contract.ts`) + the orm-client resolver. Reconcile the `parentCols/childCols` field-name drift to `parentColumns/childColumns`. Does **not** teach any consumer (read/filter/write) to use `through` — that's slices 1–3. `pnpm fixtures:check` regen is in-scope. + +### Parallel group (each builds on slice 0; mutually independent) + +- **Slice `01-correlated-read-through-junction`** — Linear: [TML-2785](https://linear.app/prisma-company/issue/TML-2785) + - **Outcome:** `db.orm.User.include('tags')` returns `{ …user, tags: Tag[] }` for an M:N relation, in a single SQL execution (one correlated subquery walking the junction, no LATERAL). + - **Builds on:** Slice 0's `ResolvedRelation.through`. + - **Hands to:** the include-projection junction-walk pattern (a reference for how filter/write traverse `through`). + - **Focus:** extend `buildCorrelatedIncludeProjection` (`query-plan-select.ts`) to correlate parent → junction → target; PG + SQLite integration tests. No LATERAL, no multi-query. + +- **Slice `02-filter-exists-through-junction`** — Linear: [TML-2786](https://linear.app/prisma-company/issue/TML-2786) + - **Outcome:** `.filter((u) => u.tags.some/every/none(...))` emits an EXISTS subquery that walks the junction for M:N relations. + - **Builds on:** Slice 0's `ResolvedRelation.through`. + - **Hands to:** correctly-shaped M:N relation filters (consumed by any query using `.some/.every/.none`). + - **Focus:** teach `buildJoinWhere` / `createRelationFilterAccessor` (`model-accessor.ts`) to add the junction hop; PG + SQLite integration. + +- **Slice `03-nested-write-through-junction`** — Linear: [TML-2787](https://linear.app/prisma-company/issue/TML-2787) + - **Outcome:** Nested `connect` / `disconnect` / `create` over M:N route to junction INSERT / DELETE under both `create()` and `update()`; nested `.create` over a required-payload junction is disabled at types **and** runtime. + - **Builds on:** Slice 0's `ResolvedRelation.through` + required-payload-column info. + - **Hands to:** the relation-shaped M:N write API (the shape the Pothos plugin wires against). + - **Focus:** remove the `partitionByOwnership()` "not supported yet" guard; route M:N as junction writes (not parent-/child-owned); flip the rejection unit test to positive; the type-level `.create` disable on required-payload junctions is its own dispatch. **Heaviest slice — re-check *Small* at `drive-plan-slice`; split the type-level disable into its own slice if it doesn't hold as one review.** + +## Dependencies (external) + +- [x] Correlated-only read path (TML-2729, PR #667) landed on `main` — slice 1 extends `buildCorrelatedIncludeProjection`, which exists. +- [x] Single-query mutation read-back (TML-2657) landed — no multi-query stitcher to reconcile. + +## Sequencing rationale + +Slice 0 is a hard gate, not a stylistic choice: until the contract validates an M:N relation and the resolver surfaces `through`, slices 1–3 have no validatable integration fixture to test against and nothing to read `through` from. Once slice 0 lands, the three consumers touch disjoint files (`query-plan-select.ts` / `model-accessor.ts` / `mutation-executor.ts`) and share only the read-only `ResolvedRelation.through` field — no write-write contention — so they parallelise cleanly. They are sequenced after 0 purely by data dependency, not by reviewer pacing. diff --git a/projects/sql-orm-many-to-many/spec.md b/projects/sql-orm-many-to-many/spec.md new file mode 100644 index 0000000000..a9887a273d --- /dev/null +++ b/projects/sql-orm-many-to-many/spec.md @@ -0,0 +1,105 @@ +# SQL ORM — Many-to-Many End to End + +> Linear Project: [SQL ORM: Many-to-Many End to End](https://linear.app/prisma-company/project/sql-orm-many-to-many-end-to-end-c178df40ca3a) · Planning record: TML-2597 (`Plan: …`) · Slices: TML-2784…2787 · Branch: `tml-2597-sql-orm-complete-end-to-end-many-to-many-support-include` + +## Purpose + +Many-to-many is the most-asked-about gap for users migrating from Prisma's relation API. This project exists so that M:N relations are **first-class in the ORM client** — readable, filterable, and writable through their junction — rather than something users must hand-wire as explicit junction models and that downstream integrations (the Pothos plugin) must reject at schema-build time. The relation-shaped API *is* the product promise; M:N is the hole in it. + +## At a glance + +Authoring already works — `rel.manyToMany('Tag', { through: 'UserTag', from: 'userId', to: 'tagId' })` lowers to a `RelationNode` with a fully populated `through`. The runtime ORM is where it falls apart. + +**Today** the relation-shaped API is unavailable for M:N at every step: + +```ts +db.orm.User.include('tags') // ✗ resolver reads on.localFields only; ignores `through` +db.orm.User.filter((u) => u.tags.some(/* … */)) // ✗ EXISTS skips the junction → wrong shape +db.orm.User.create({ tags: (t) => t.connect({ id }) }) // ✗ throws "M:N nested mutations are not supported yet" +``` + +…and, discovered while shaping this project: **an emitted M:N contract does not even validate.** `validateContract` rejects it two ways — `cardinality: 'N:M'` isn't in the relation validator's enum, and `through` is an undeclared key on a reject-policy object. So there is no validatable end-to-end M:N contract at all, which is *why* downstream integrations route users to explicit junction models. + +**After this project:** + +```ts +db.orm.User.include('tags') // → { …user, tags: Tag[] } (one correlated query through the junction) +db.orm.User.filter((u) => u.tags.some((t) => t.name.eq('x'))) // → EXISTS walking the junction +db.orm.User.update({ tags: (t) => t.connect({ id }) }) // → junction INSERT +db.orm.User.update({ tags: (t) => t.disconnect({ id }) }) // → junction DELETE +``` + +The whole project hangs off one new primitive: a uniform **`through` descriptor** on the resolved relation (`parent → junction → target`), surfaced once by the single shared resolver and consumed three ways — correlated read, EXISTS filter, junction-DML write. + +## Non-goals + +- **New nested-write kinds.** `update` / `updateMany` / `upsert` / `delete` / `set` / `connectOrCreate` on related rows do not exist for *any* cardinality today and are **out of scope** — tracked separately in [TML-2781](https://linear.app/prisma-company/issue/TML-2781). This project ships only the three kinds that exist: `create` / `connect` / `disconnect`. +- **Junction payload columns through the M:N sugar.** Reading or writing non-FK columns on the junction table is served by the junction model's own 1:N relations + the SQL query builder. The M:N sugar deliberately does not expose them. +- **Auto-synthesised junction models.** `through` must reference an authored model (lowering throws otherwise); this project does not generate implicit junctions. +- **LATERAL / multi-query read strategies.** Removed by TML-2729 / TML-2657; the read path is correlated-only and stays that way. +- **Wiring the Pothos plugin itself.** This project makes the runtime M:N API shape match what the plugin needs (`{ connect: { id } }` / `{ disconnect: { id } }`); the plugin-side wiring is downstream. + +## Place in the larger world + +- **sql-orm-client** (`packages/3-extensions/sql-orm-client`) is an optional extension over the SQL contract + query lane — ADR 015 (ORM as Optional Extension). +- **Contract surface.** The `through` extension lives in the SQL domain relation format — ADR 172 (Contract domain-storage separation) and ADR 121 (Contract.d.ts structure and relation typing) constrain its shape. The cast at `build-contract.ts` flagged "until the contract type is extended to cover many-to-many" is the seam this project closes. +- **Builds on** the correlated-only read path (TML-2729) and the single-query mutation read-back (TML-2657) — there is no LATERAL or multi-query fallback to extend. +- **Primary downstream consumer:** the Pothos plugin, which today rejects M:N at schema build. The runtime callback shapes must match what plugin-prisma users expect so the plugin can do the obvious wiring. + +## Cross-cutting requirements + +- **An M:N contract must emit and round-trip through `validateContract`.** This is the foundation every slice's integration fixture depends on — no validatable M:N contract exists today. (System-level because it's a prerequisite shared by all three consumer slices, not owned by any one of them.) +- **The junction-walk is a single shared primitive.** The `through` descriptor is surfaced once, through the one `resolveModelRelations` → `ResolvedRelation` resolver that already feeds includes, filters, and mutations. Slices consume it differently but must not fork the resolution. +- **Cardinality tag canonicalised on `'N:M'` repo-wide.** The contract, schema, PSL, and lowering already use `'N:M'`; the orm-client's lone `'M:N'` spelling is reconciled to it (not translated at a boundary). No `'M:N'`/`'N:M'` split survives. +- **PG + SQLite integration coverage** for every user-observable M:N path (read, filter, write). +- **Every merged slice leaves non-M:N paths green and the system deployable.** M:N support arrives incrementally; partial support must never regress existing cardinalities. + +## Transitional-shape constraints + +- **Slice 0 is a contract-shape change (hash change).** From slice 0 onward, emitted fixtures/goldens carry `through` + `N:M`; `pnpm fixtures:check` must be green at every slice boundary, and unrelated fixture drift is investigated, not committed. +- **The change is purely additive to the contract** — existing non-M:N contracts are unchanged, so no deprecation window is required; but each slice must keep CI green on the project working branch before the next builds on it. + +## Project Definition of Done + +Inherits the team-DoD floor ([`drive/calibration/dod.md`](../../drive/calibration/dod.md)) — not restated here. Project-specific conditions on top: + +- [ ] An M:N contract (`rel.manyToMany` with `through`) emits and round-trips `validateContract` (fails today). +- [ ] `db.orm.User.include('tags')` returns `{ …user, tags: Tag[] }` in a **single** SQL execution (one correlated subquery through the junction, no LATERAL) — PG + SQLite integration tests. +- [ ] `.filter((u) => u.tags.some/every/none(...))` emits an EXISTS that walks the junction — PG + SQLite. +- [ ] Nested `connect` / `disconnect` / `create` over M:N route to junction INSERT / DELETE, under both `create()` and `update()` parent flows; the `partitionByOwnership()` "not supported yet" guard is removed and its unit test flips to a positive assertion. +- [ ] Nested `.create` over a **required-payload junction** (junction with required non-FK columns) is disabled **at types AND at runtime**, with a message pointing to the junction model's 1:N relations / the SQL builder. (No-required-payload junctions allow ergonomic nested `create`.) +- [ ] The runtime M:N callback shape matches what the Pothos plugin needs (`{ connect: { id } }` / `{ disconnect: { id } }`). +- [ ] ADR for the `through` relation contract extension authored or ADR 121 amended (see ADR pointer). + +## Contract-impact + +_Required: this project changes the contract surface._ + +- **Entities affected:** the SQL domain relation — the `ModelRelation` JSON schema (`data-contract-sql-v1.json`), the `ContractReferenceRelation` arktype validator (`packages/2-sql/1-core/contract`), and the corresponding `ContractReferenceRelation` TS type (`@prisma-next/sql-contract/types`). +- **New / changed kinds:** relation `cardinality` enum gains `'N:M'`; relation gains optional `through: { table, parentColumns, childColumns }` (canonical field names match lowering; `build-contract`'s `parentCols/childCols` drift is reconciled). The `as ContractRelation['cardinality']` cast in `build-contract.ts` is deleted. +- **Migration plan for downstream consumers:** purely **additive** — existing non-M:N contracts are byte-unchanged in shape, so no consumer breaks and no deprecation window is needed. The contract **hash** changes, so all emitted fixtures/goldens regenerate; `pnpm fixtures:check` gates this. `validateContract` consumers gain M:N acceptance (today they reject it), so this only widens what validates. + +## Adapter-impact + +_Required: confirm target-adapter reach._ + +- **No adapter code changes** (`packages/3-targets/**` untouched). The work is in the contract + the sql-orm-client extension; SQL is produced by the existing renderer/query lane. +- **postgres + sqlite** must execute the new shapes: correlated junction subquery (read), EXISTS-through-junction (filter), junction INSERT/DELETE (write). Both are covered by integration tests (PGlite + SQLite). +- **mongo:** N/A — SQL ORM only. + +## ADR pointer + +The `through` extension to the relation contract is a durable contract-surface change. Working position: **amend ADR 121** (Contract.d.ts structure and relation typing) to cover the M:N relation shape (`through` + `N:M` cardinality), cross-referencing ADR 172. Confirm at close-out whether an amendment suffices or a standalone ADR is warranted (per the ADR-audit DoD item). + +## Open Questions + +1. **Cardinality canonicalisation blast radius.** Canonicalising on `'N:M'` touches the orm-client's `RelationCardinalityTag`. Working position: it's contained to sql-orm-client; slice 0 greps for any other `'M:N'` hardcode and reconciles or documents it. + +_Resolved during shaping (2026-06-01): project slug is `sql-orm-many-to-many`; promoted to a dedicated Linear Project ([SQL ORM: Many-to-Many End to End](https://linear.app/prisma-company/project/sql-orm-many-to-many-end-to-end-c178df40ca3a)) — TML-2597 became the `Plan: …` record (Done); four slice issues TML-2784…2787 carry the work._ + +## References + +- Linear Project: [SQL ORM: Many-to-Many End to End](https://linear.app/prisma-company/project/sql-orm-many-to-many-end-to-end-c178df40ca3a) · planning record TML-2597 · slices TML-2784 (0) / TML-2785 (1) / TML-2786 (2) / TML-2787 (3) +- Related / deferred: [TML-2781](https://linear.app/prisma-company/issue/TML-2781) (new nested-write kinds), [TML-2729](https://linear.app/prisma-company/issue/TML-2729) (correlated-only reads), [TML-2657](https://linear.app/prisma-company/issue/TML-2657) (single-query mutation read-back) +- ADRs: ADR 015 (ORM as Optional Extension), ADR 121 (Contract.d.ts structure and relation typing), ADR 172 (Contract domain-storage separation) +- Design-discussion record: this session (`drive-discussion`, 2026-06-01) — Option A representation, 4-slice shape, write scope + required-payload guard diff --git a/projects/sql-orm-many-to-many/trace.jsonl b/projects/sql-orm-many-to-many/trace.jsonl new file mode 100644 index 0000000000..613f6f7c9f --- /dev/null +++ b/projects/sql-orm-many-to-many/trace.jsonl @@ -0,0 +1,5 @@ +{"event_id":"63c4bb68-1e43-4d8a-ae8a-0cf5a673b87a","schema_version":"1","ts":"2026-06-01T15:59:16.324Z","project_run_id":"sql-orm-many-to-many","orchestrator_agent_id":null,"event_type":"spec-authored","spec_path":"projects/sql-orm-many-to-many/spec.md","spec_kind":"project","byte_length":11050,"edge_cases_count":null,"open_questions_count":3,"dod_items_count":7} +{"event_id":"27b905e9-bd7a-461e-8747-13899d114f84","schema_version":"1","ts":"2026-06-01T16:09:49.146Z","project_run_id":"sql-orm-many-to-many","orchestrator_agent_id":null,"event_type":"spec-amended","spec_path":"projects/sql-orm-many-to-many/spec.md","spec_kind":"project","byte_length":10740,"bytes_delta":-310,"edge_cases_count":null,"open_questions_count":1,"dod_items_count":7,"reason":"operator-correction","sections_changed":["Open Questions"]} +{"event_id":"fe46c844-0bde-4718-ad08-3a2f96ee44bc","schema_version":"1","ts":"2026-06-01T16:12:42.699Z","project_run_id":"sql-orm-many-to-many","orchestrator_agent_id":null,"event_type":"plan-amended","plan_path":"projects/sql-orm-many-to-many/plan.md","plan_kind":"project","byte_length":4874,"bytes_delta":4214,"slice_count":4,"dispatch_count":null,"dispatch_size_distribution":null,"open_items_count":1,"reason":"replan-from-discussion","dispatches_added":null,"dispatches_removed":null,"dispatches_resized":null} +{"event_id":"4b5eb287-2c5e-4e4f-b05a-3c317d1a1f89","schema_version":"1","ts":"2026-06-01T16:20:11.289Z","project_run_id":"sql-orm-many-to-many","orchestrator_agent_id":null,"event_type":"plan-amended","plan_path":"projects/sql-orm-many-to-many/plan.md","plan_kind":"project","byte_length":5062,"bytes_delta":188,"slice_count":4,"dispatch_count":null,"dispatch_size_distribution":null,"open_items_count":0,"reason":"operator-correction","dispatches_added":null,"dispatches_removed":null,"dispatches_resized":null} +{"event_id":"010ef0b7-c98c-4ad3-9b8b-9b146370d8b7","schema_version":"1","ts":"2026-06-01T16:20:11.812Z","project_run_id":"sql-orm-many-to-many","orchestrator_agent_id":null,"event_type":"spec-amended","spec_path":"projects/sql-orm-many-to-many/spec.md","spec_kind":"project","byte_length":11054,"bytes_delta":314,"edge_cases_count":null,"open_questions_count":1,"dod_items_count":7,"reason":"operator-correction","sections_changed":["header","Open Questions","References"]}