Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions projects/sql-orm-many-to-many/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# SQL ORM — Many-to-Many End to End

Transient project workspace for [TML-2597](https://linear.app/prisma-company/issue/TML-2597/sql-orm-complete-end-to-end-many-to-many-support-include-path-nested). See [`spec.md`](./spec.md) for the project spec and [`plan.md`](./plan.md) for the slice sequencing.

Branch: `tml-2597-sql-orm-complete-end-to-end-many-to-many-support-include`.

> Everything under `projects/` is transient — migrated to `docs/` or deleted at close-out per [`projects/README.md`](../README.md).
47 changes: 47 additions & 0 deletions projects/sql-orm-many-to-many/plan.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# SQL ORM — Many-to-Many End to End — Plan

**Spec:** `projects/sql-orm-many-to-many/spec.md`
**Linear Project:** [SQL ORM: Many-to-Many End to End](https://linear.app/prisma-company/project/sql-orm-many-to-many-end-to-end-c178df40ca3a) (planning record: TML-2597, `Plan: …`, Done)

## At a glance

Four slices: one **foundation slice** (slice 0) that makes M:N a validatable contract shape and surfaces the shared `through` descriptor, then a **three-way parallel fan-out** — read, filter, write — each consuming slice 0's hand-off and independent of the others.

## Composition

### Stack (deliver in order)

1. **Slice `00-contract-resolver-foundation`** — Linear: [TML-2784](https://linear.app/prisma-company/issue/TML-2784)
- **Outcome:** An M:N relation (`rel.manyToMany` with `through`) emits a contract that round-trips `validateContract`; the shared resolver surfaces a uniform `through` descriptor (and the junction's required-non-FK-column info); the cardinality tag is canonicalised on `'N:M'` repo-wide.
- **Builds on:** None (correlated-only read path from TML-2729 / TML-2657 already on `main`).
- **Hands to:** (a) a validatable M:N contract shape — `through: { table, parentColumns, childColumns }` declared in the JSON schema + arktype validator + `ContractReferenceRelation` type; (b) `ResolvedRelation.through` + required-payload-column info on `resolveModelRelations`; (c) a single `'N:M'` cardinality tag with no `'M:N'` left in sql-orm-client.
- **Focus:** contract surface (`packages/2-sql/1-core/contract` validator, `data-contract-sql-v1.json`, `ContractReferenceRelation` type, delete the `as ContractRelation['cardinality']` cast in `build-contract.ts`) + the orm-client resolver. Reconcile the `parentCols/childCols` field-name drift to `parentColumns/childColumns`. Does **not** teach any consumer (read/filter/write) to use `through` — that's slices 1–3. `pnpm fixtures:check` regen is in-scope.

### Parallel group (each builds on slice 0; mutually independent)

- **Slice `01-correlated-read-through-junction`** — Linear: [TML-2785](https://linear.app/prisma-company/issue/TML-2785)
- **Outcome:** `db.orm.User.include('tags')` returns `{ …user, tags: Tag[] }` for an M:N relation, in a single SQL execution (one correlated subquery walking the junction, no LATERAL).
- **Builds on:** Slice 0's `ResolvedRelation.through`.
- **Hands to:** the include-projection junction-walk pattern (a reference for how filter/write traverse `through`).
- **Focus:** extend `buildCorrelatedIncludeProjection` (`query-plan-select.ts`) to correlate parent → junction → target; PG + SQLite integration tests. No LATERAL, no multi-query.

- **Slice `02-filter-exists-through-junction`** — Linear: [TML-2786](https://linear.app/prisma-company/issue/TML-2786)
- **Outcome:** `.filter((u) => u.tags.some/every/none(...))` emits an EXISTS subquery that walks the junction for M:N relations.
- **Builds on:** Slice 0's `ResolvedRelation.through`.
- **Hands to:** correctly-shaped M:N relation filters (consumed by any query using `.some/.every/.none`).
- **Focus:** teach `buildJoinWhere` / `createRelationFilterAccessor` (`model-accessor.ts`) to add the junction hop; PG + SQLite integration.

- **Slice `03-nested-write-through-junction`** — Linear: [TML-2787](https://linear.app/prisma-company/issue/TML-2787)
- **Outcome:** Nested `connect` / `disconnect` / `create` over M:N route to junction INSERT / DELETE under both `create()` and `update()`; nested `.create` over a required-payload junction is disabled at types **and** runtime.
- **Builds on:** Slice 0's `ResolvedRelation.through` + required-payload-column info.
- **Hands to:** the relation-shaped M:N write API (the shape the Pothos plugin wires against).
- **Focus:** remove the `partitionByOwnership()` "not supported yet" guard; route M:N as junction writes (not parent-/child-owned); flip the rejection unit test to positive; the type-level `.create` disable on required-payload junctions is its own dispatch. **Heaviest slice — re-check *Small* at `drive-plan-slice`; split the type-level disable into its own slice if it doesn't hold as one review.**

## Dependencies (external)

- [x] Correlated-only read path (TML-2729, PR #667) landed on `main` — slice 1 extends `buildCorrelatedIncludeProjection`, which exists.
- [x] Single-query mutation read-back (TML-2657) landed — no multi-query stitcher to reconcile.

## Sequencing rationale

Slice 0 is a hard gate, not a stylistic choice: until the contract validates an M:N relation and the resolver surfaces `through`, slices 1–3 have no validatable integration fixture to test against and nothing to read `through` from. Once slice 0 lands, the three consumers touch disjoint files (`query-plan-select.ts` / `model-accessor.ts` / `mutation-executor.ts`) and share only the read-only `ResolvedRelation.through` field — no write-write contention — so they parallelise cleanly. They are sequenced after 0 purely by data dependency, not by reviewer pacing.
105 changes: 105 additions & 0 deletions projects/sql-orm-many-to-many/spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# SQL ORM — Many-to-Many End to End

> Linear Project: [SQL ORM: Many-to-Many End to End](https://linear.app/prisma-company/project/sql-orm-many-to-many-end-to-end-c178df40ca3a) · Planning record: TML-2597 (`Plan: …`) · Slices: TML-2784…2787 · Branch: `tml-2597-sql-orm-complete-end-to-end-many-to-many-support-include`

## Purpose

Many-to-many is the most-asked-about gap for users migrating from Prisma's relation API. This project exists so that M:N relations are **first-class in the ORM client** — readable, filterable, and writable through their junction — rather than something users must hand-wire as explicit junction models and that downstream integrations (the Pothos plugin) must reject at schema-build time. The relation-shaped API *is* the product promise; M:N is the hole in it.

## At a glance

Authoring already works — `rel.manyToMany('Tag', { through: 'UserTag', from: 'userId', to: 'tagId' })` lowers to a `RelationNode` with a fully populated `through`. The runtime ORM is where it falls apart.

**Today** the relation-shaped API is unavailable for M:N at every step:

```ts
db.orm.User.include('tags') // ✗ resolver reads on.localFields only; ignores `through`
db.orm.User.filter((u) => u.tags.some(/* … */)) // ✗ EXISTS skips the junction → wrong shape
db.orm.User.create({ tags: (t) => t.connect({ id }) }) // ✗ throws "M:N nested mutations are not supported yet"
```

…and, discovered while shaping this project: **an emitted M:N contract does not even validate.** `validateContract` rejects it two ways — `cardinality: 'N:M'` isn't in the relation validator's enum, and `through` is an undeclared key on a reject-policy object. So there is no validatable end-to-end M:N contract at all, which is *why* downstream integrations route users to explicit junction models.

**After this project:**

```ts
db.orm.User.include('tags') // → { …user, tags: Tag[] } (one correlated query through the junction)
db.orm.User.filter((u) => u.tags.some((t) => t.name.eq('x'))) // → EXISTS walking the junction
db.orm.User.update({ tags: (t) => t.connect({ id }) }) // → junction INSERT
db.orm.User.update({ tags: (t) => t.disconnect({ id }) }) // → junction DELETE
```

The whole project hangs off one new primitive: a uniform **`through` descriptor** on the resolved relation (`parent → junction → target`), surfaced once by the single shared resolver and consumed three ways — correlated read, EXISTS filter, junction-DML write.

## Non-goals

- **New nested-write kinds.** `update` / `updateMany` / `upsert` / `delete` / `set` / `connectOrCreate` on related rows do not exist for *any* cardinality today and are **out of scope** — tracked separately in [TML-2781](https://linear.app/prisma-company/issue/TML-2781). This project ships only the three kinds that exist: `create` / `connect` / `disconnect`.
- **Junction payload columns through the M:N sugar.** Reading or writing non-FK columns on the junction table is served by the junction model's own 1:N relations + the SQL query builder. The M:N sugar deliberately does not expose them.
- **Auto-synthesised junction models.** `through` must reference an authored model (lowering throws otherwise); this project does not generate implicit junctions.
- **LATERAL / multi-query read strategies.** Removed by TML-2729 / TML-2657; the read path is correlated-only and stays that way.
- **Wiring the Pothos plugin itself.** This project makes the runtime M:N API shape match what the plugin needs (`{ connect: { id } }` / `{ disconnect: { id } }`); the plugin-side wiring is downstream.

## Place in the larger world

- **sql-orm-client** (`packages/3-extensions/sql-orm-client`) is an optional extension over the SQL contract + query lane — ADR 015 (ORM as Optional Extension).
- **Contract surface.** The `through` extension lives in the SQL domain relation format — ADR 172 (Contract domain-storage separation) and ADR 121 (Contract.d.ts structure and relation typing) constrain its shape. The cast at `build-contract.ts` flagged "until the contract type is extended to cover many-to-many" is the seam this project closes.
- **Builds on** the correlated-only read path (TML-2729) and the single-query mutation read-back (TML-2657) — there is no LATERAL or multi-query fallback to extend.
- **Primary downstream consumer:** the Pothos plugin, which today rejects M:N at schema build. The runtime callback shapes must match what plugin-prisma users expect so the plugin can do the obvious wiring.

## Cross-cutting requirements

- **An M:N contract must emit and round-trip through `validateContract`.** This is the foundation every slice's integration fixture depends on — no validatable M:N contract exists today. (System-level because it's a prerequisite shared by all three consumer slices, not owned by any one of them.)
- **The junction-walk is a single shared primitive.** The `through` descriptor is surfaced once, through the one `resolveModelRelations` → `ResolvedRelation` resolver that already feeds includes, filters, and mutations. Slices consume it differently but must not fork the resolution.
- **Cardinality tag canonicalised on `'N:M'` repo-wide.** The contract, schema, PSL, and lowering already use `'N:M'`; the orm-client's lone `'M:N'` spelling is reconciled to it (not translated at a boundary). No `'M:N'`/`'N:M'` split survives.
- **PG + SQLite integration coverage** for every user-observable M:N path (read, filter, write).
- **Every merged slice leaves non-M:N paths green and the system deployable.** M:N support arrives incrementally; partial support must never regress existing cardinalities.

## Transitional-shape constraints

- **Slice 0 is a contract-shape change (hash change).** From slice 0 onward, emitted fixtures/goldens carry `through` + `N:M`; `pnpm fixtures:check` must be green at every slice boundary, and unrelated fixture drift is investigated, not committed.
- **The change is purely additive to the contract** — existing non-M:N contracts are unchanged, so no deprecation window is required; but each slice must keep CI green on the project working branch before the next builds on it.

## Project Definition of Done

Inherits the team-DoD floor ([`drive/calibration/dod.md`](../../drive/calibration/dod.md)) — not restated here. Project-specific conditions on top:

- [ ] An M:N contract (`rel.manyToMany` with `through`) emits and round-trips `validateContract` (fails today).
- [ ] `db.orm.User.include('tags')` returns `{ …user, tags: Tag[] }` in a **single** SQL execution (one correlated subquery through the junction, no LATERAL) — PG + SQLite integration tests.
- [ ] `.filter((u) => u.tags.some/every/none(...))` emits an EXISTS that walks the junction — PG + SQLite.
- [ ] Nested `connect` / `disconnect` / `create` over M:N route to junction INSERT / DELETE, under both `create()` and `update()` parent flows; the `partitionByOwnership()` "not supported yet" guard is removed and its unit test flips to a positive assertion.
- [ ] Nested `.create` over a **required-payload junction** (junction with required non-FK columns) is disabled **at types AND at runtime**, with a message pointing to the junction model's 1:N relations / the SQL builder. (No-required-payload junctions allow ergonomic nested `create`.)
- [ ] The runtime M:N callback shape matches what the Pothos plugin needs (`{ connect: { id } }` / `{ disconnect: { id } }`).
- [ ] ADR for the `through` relation contract extension authored or ADR 121 amended (see ADR pointer).

## Contract-impact

_Required: this project changes the contract surface._

- **Entities affected:** the SQL domain relation — the `ModelRelation` JSON schema (`data-contract-sql-v1.json`), the `ContractReferenceRelation` arktype validator (`packages/2-sql/1-core/contract`), and the corresponding `ContractReferenceRelation` TS type (`@prisma-next/sql-contract/types`).
- **New / changed kinds:** relation `cardinality` enum gains `'N:M'`; relation gains optional `through: { table, parentColumns, childColumns }` (canonical field names match lowering; `build-contract`'s `parentCols/childCols` drift is reconciled). The `as ContractRelation['cardinality']` cast in `build-contract.ts` is deleted.
- **Migration plan for downstream consumers:** purely **additive** — existing non-M:N contracts are byte-unchanged in shape, so no consumer breaks and no deprecation window is needed. The contract **hash** changes, so all emitted fixtures/goldens regenerate; `pnpm fixtures:check` gates this. `validateContract` consumers gain M:N acceptance (today they reject it), so this only widens what validates.

## Adapter-impact

_Required: confirm target-adapter reach._

- **No adapter code changes** (`packages/3-targets/**` untouched). The work is in the contract + the sql-orm-client extension; SQL is produced by the existing renderer/query lane.
- **postgres + sqlite** must execute the new shapes: correlated junction subquery (read), EXISTS-through-junction (filter), junction INSERT/DELETE (write). Both are covered by integration tests (PGlite + SQLite).
- **mongo:** N/A — SQL ORM only.

## ADR pointer

The `through` extension to the relation contract is a durable contract-surface change. Working position: **amend ADR 121** (Contract.d.ts structure and relation typing) to cover the M:N relation shape (`through` + `N:M` cardinality), cross-referencing ADR 172. Confirm at close-out whether an amendment suffices or a standalone ADR is warranted (per the ADR-audit DoD item).

## Open Questions

1. **Cardinality canonicalisation blast radius.** Canonicalising on `'N:M'` touches the orm-client's `RelationCardinalityTag`. Working position: it's contained to sql-orm-client; slice 0 greps for any other `'M:N'` hardcode and reconciles or documents it.

_Resolved during shaping (2026-06-01): project slug is `sql-orm-many-to-many`; promoted to a dedicated Linear Project ([SQL ORM: Many-to-Many End to End](https://linear.app/prisma-company/project/sql-orm-many-to-many-end-to-end-c178df40ca3a)) — TML-2597 became the `Plan: …` record (Done); four slice issues TML-2784…2787 carry the work._

## References

- Linear Project: [SQL ORM: Many-to-Many End to End](https://linear.app/prisma-company/project/sql-orm-many-to-many-end-to-end-c178df40ca3a) · planning record TML-2597 · slices TML-2784 (0) / TML-2785 (1) / TML-2786 (2) / TML-2787 (3)
- Related / deferred: [TML-2781](https://linear.app/prisma-company/issue/TML-2781) (new nested-write kinds), [TML-2729](https://linear.app/prisma-company/issue/TML-2729) (correlated-only reads), [TML-2657](https://linear.app/prisma-company/issue/TML-2657) (single-query mutation read-back)
- ADRs: ADR 015 (ORM as Optional Extension), ADR 121 (Contract.d.ts structure and relation typing), ADR 172 (Contract domain-storage separation)
- Design-discussion record: this session (`drive-discussion`, 2026-06-01) — Option A representation, 4-slice shape, write scope + required-payload guard
5 changes: 5 additions & 0 deletions projects/sql-orm-many-to-many/trace.jsonl
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{"event_id":"63c4bb68-1e43-4d8a-ae8a-0cf5a673b87a","schema_version":"1","ts":"2026-06-01T15:59:16.324Z","project_run_id":"sql-orm-many-to-many","orchestrator_agent_id":null,"event_type":"spec-authored","spec_path":"projects/sql-orm-many-to-many/spec.md","spec_kind":"project","byte_length":11050,"edge_cases_count":null,"open_questions_count":3,"dod_items_count":7}
{"event_id":"27b905e9-bd7a-461e-8747-13899d114f84","schema_version":"1","ts":"2026-06-01T16:09:49.146Z","project_run_id":"sql-orm-many-to-many","orchestrator_agent_id":null,"event_type":"spec-amended","spec_path":"projects/sql-orm-many-to-many/spec.md","spec_kind":"project","byte_length":10740,"bytes_delta":-310,"edge_cases_count":null,"open_questions_count":1,"dod_items_count":7,"reason":"operator-correction","sections_changed":["Open Questions"]}
{"event_id":"fe46c844-0bde-4718-ad08-3a2f96ee44bc","schema_version":"1","ts":"2026-06-01T16:12:42.699Z","project_run_id":"sql-orm-many-to-many","orchestrator_agent_id":null,"event_type":"plan-amended","plan_path":"projects/sql-orm-many-to-many/plan.md","plan_kind":"project","byte_length":4874,"bytes_delta":4214,"slice_count":4,"dispatch_count":null,"dispatch_size_distribution":null,"open_items_count":1,"reason":"replan-from-discussion","dispatches_added":null,"dispatches_removed":null,"dispatches_resized":null}
{"event_id":"4b5eb287-2c5e-4e4f-b05a-3c317d1a1f89","schema_version":"1","ts":"2026-06-01T16:20:11.289Z","project_run_id":"sql-orm-many-to-many","orchestrator_agent_id":null,"event_type":"plan-amended","plan_path":"projects/sql-orm-many-to-many/plan.md","plan_kind":"project","byte_length":5062,"bytes_delta":188,"slice_count":4,"dispatch_count":null,"dispatch_size_distribution":null,"open_items_count":0,"reason":"operator-correction","dispatches_added":null,"dispatches_removed":null,"dispatches_resized":null}
{"event_id":"010ef0b7-c98c-4ad3-9b8b-9b146370d8b7","schema_version":"1","ts":"2026-06-01T16:20:11.812Z","project_run_id":"sql-orm-many-to-many","orchestrator_agent_id":null,"event_type":"spec-amended","spec_path":"projects/sql-orm-many-to-many/spec.md","spec_kind":"project","byte_length":11054,"bytes_delta":314,"edge_cases_count":null,"open_questions_count":1,"dod_items_count":7,"reason":"operator-correction","sections_changed":["header","Open Questions","References"]}