Skip to content

Commit ee58133

Browse files
Merge pull request #206 from datajoint/docs/v2.3.1-cohesion
docs: fold 2.3.1 changes into What's New in 2.3 + fix cross-link gaps
2 parents 607b381 + cd90aa3 commit ee58133

3 files changed

Lines changed: 12 additions & 1 deletion

File tree

src/about/whats-new-23.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,14 @@ DataJoint 2.3 adds a first-class **upstream read surface** — `Diagram.trace` a
66
77
> **Citation:** Yatsenko D, Nguyen TT. *DataJoint 2.0: A Computational Substrate for Agentic Scientific Workflows.* arXiv:2602.16585. 2026. [doi:10.48550/arXiv.2602.16585](https://doi.org/10.48550/arXiv.2602.16585)
88
9+
## Changes in 2.3.1
10+
11+
2.3.1 is a patch release on the 2.3 line. If you are upgrading from **2.3.0**:
12+
13+
- **`strict_provenance` removed.** The opt-in `dj.config["strict_provenance"]` runtime guardrail that shipped in 2.3.0 has been retired. Comprehensively checking the `make()` read/write contract across every access path is a code-inspection problem rather than a runtime one, so it is validated at review or deploy time rather than by an in-process flag. The flag was off by default with no known adoption, so existing pipelines are unaffected. The ergonomic read surface — `Diagram.trace` and `self.upstream` — is unchanged, and the rules are documented as the [make() reproducibility contract](../reference/specs/autopopulate.md#43-the-make-reproducibility-contract).
14+
- **Python 3.14 support.** `requires-python` now spans `>=3.10,<3.15`; CI exercises both ends of the range (3.10 and 3.14).
15+
- **Garbage-collection fix.** `gc.collect()` now discovers codec-referenced files correctly, closing a path where live custom-codec and schema-addressed object-store files (`<object@>`, `<npy@>`) could be misclassified as orphans and deleted. See [Clean Up Object Storage](../how-to/garbage-collection.md).
16+
917
## Overview
1018

1119
A computed row is reproducible only when `make(self, key)` reads only from its declared upstream dependencies and writes only to `self` (and its Parts). DataJoint 2.3 makes that read/write boundary easy to follow — and the resulting data lineage easy to query — with two features designed as a unit:

src/explanation/computation-model.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -451,6 +451,7 @@ MyTable.populate(key, max_calls=1)
451451
**Specifications**
452452

453453
- [AutoPopulate](../reference/specs/autopopulate.md) — normative spec for `key_source`, the [make() reproducibility contract](../reference/specs/autopopulate.md#43-the-make-reproducibility-contract), the tripartite pattern, and job reservation.
454+
- [Upstream Trace](../reference/specs/trace.md)`Diagram.trace` and `self.upstream`, the read surface for declared upstream inside `make()`.
454455
- [Cascade](../reference/specs/cascade.md) — restriction propagation and master–part integrity for cascading deletes.
455456
- [Diagram](../reference/specs/diagram.md) — the dependency graph that `populate()` and `key_source` are computed from.
456457

@@ -463,3 +464,5 @@ MyTable.populate(key, max_calls=1)
463464

464465
- [Relational workflow model](relational-workflow-model.md) — how computation fits DataJoint's data model.
465466
- [Data pipelines](data-pipelines.md) — the pipeline abstraction that auto-populated tables extend.
467+
- [Fan-out ingestion](fan-out-ingestion.md) — one ingestion step populating several entry-point tables, and where origin is recorded.
468+
- [Comparison to provenance systems](comparison-to-provenance-systems.md) — how DataJoint's structural lineage relates to dedicated provenance tooling.

src/reference/specs/autopopulate.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -310,7 +310,7 @@ The value of an auto-populated table is that every one of its rows is a *reprodu
310310

311311
3. **Read only upstream, restricted to the job.** A `make(key)` fetches only from tables upstream of the target — the declared ancestors reachable by following the foreign keys embedded in the primary key — and only the rows reachable from the current `key`. This key-restricted set of ancestors is the entity's *upstream cone*, exposed as `self.upstream` (see [§4.4](#44-accessing-source-data)). Reading anything else — a table that is not an ancestor, or entities under a different `key` — breaks the contract and makes the result depend on data the pipeline does not track.
312312

313-
4. **Write the result to `self` and its Parts.** The derived result is inserted into `self` and the table's Part tables, atomically, as one unit. A `make()` *may* also write to other destinations — other Manual tables, files, external systems (the *fan-out* pattern) — but such writes leave the pipeline's dependency graph, so the `make()` must record the source identity at each destination (as it would at any Manual-table entry point) to keep that data traceable.
313+
4. **Write the result to `self` and its Parts.** The derived result is inserted into `self` and the table's Part tables, atomically, as one unit. A `make()` *may* also write to other destinations — other Manual tables, files, external systems (the [fan-out ingestion pattern](../../explanation/fan-out-ingestion.md)) — but such writes leave the pipeline's dependency graph, so the `make()` must record the source identity at each destination (as it would at any Manual-table entry point) to keep that data traceable.
314314

315315
5. **No other result-affecting input.** Anything that changes *what* is computed must enter through a declared upstream table (typically a Lookup or Manual parameter table), so it is part of the dependency graph. `make_kwargs` forwarded through `populate(make_kwargs=...)` may *orchestrate* a run (batching, parallelism, logging) but must never *parameterize* the result — see [§4.6](#46-additional-make-arguments).
316316

0 commit comments

Comments
 (0)