Skip to content

Commit 90da9a6

Browse files
Introduce the concept of "Workflow Ingegrity"
1 parent 883b8a7 commit 90da9a6

File tree

1 file changed

+28
-6
lines changed

1 file changed

+28
-6
lines changed

book/30-database-design/012-integrity.md

Lines changed: 28 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ Relational databases excel at expressing and enforcing such rules through **inte
5151

5252
# Types of Data Integrity Constraints
5353

54-
This section introduces six fundamental types of integrity constraints. Each will be covered in detail in subsequent chapters, with DataJoint implementation examples.
54+
This section introduces seven fundamental types of integrity constraints. Each will be covered in detail in subsequent chapters, with DataJoint implementation examples.
5555

5656
## 1. Domain Integrity
5757
**Ensures values are within valid ranges and types.**
@@ -73,13 +73,12 @@ Domain integrity restricts attribute values to predefined valid sets using:
7373
Completeness prevents missing values that could invalidate analyses:
7474
- **Required fields** cannot be left empty (non-nullable)
7575
- **Default values** provide sensible fallbacks
76-
- **NOT NULL constraints** enforce data presence
7776

7877
**Example:** Every experiment must have a start date.
7978

8079
**Covered in:**
8180
- [Tables](015-table.ipynb) — Required vs. optional attributes
82-
- [Default Values](020-default-values.ipynb) — Handling optional data
81+
- [Default Values](070-default-values.ipynb) — Handling optional data
8382

8483
---
8584

@@ -97,7 +96,7 @@ Entity integrity ensures a **one-to-one correspondence** between real-world enti
9796

9897
**Covered in:**
9998
- [Primary Keys](020-primary-key.md) — Entity integrity and the 1:1 correspondence guarantee (elaborated in detail)
100-
- [UUID](030-uuid.ipynb) — Universally unique identifiers
99+
- [UUID](025-uuid.ipynb) — Universally unique identifiers
101100

102101
---
103102

@@ -112,7 +111,7 @@ Referential integrity maintains logical associations across tables:
112111
**Example:** A recording session cannot reference a non-existent mouse.
113112

114113
**Covered in:**
115-
- [Foreign Keys](035-foreign-keys.ipynb) — Cross-table relationships
114+
- [Foreign Keys](030-foreign-keys.ipynb) — Cross-table relationships
116115
- [Relationships](050-relationships.ipynb) — Dependency patterns
117116

118117
---
@@ -147,6 +146,28 @@ Consistency provides a unified view during concurrent access:
147146
- [Concurrency](../operations/050-concurrency.ipynb) — Multi-user operations
148147
- [Transactions](../operations/045-transactions.ipynb) — ACID guarantees
149148

149+
---
150+
151+
## 7. Workflow Integrity
152+
**Operations execute in valid sequences that respect enterprise processes.**
153+
154+
Workflow integrity extends referential integrity by enforcing not just *what* relationships exist, but *when* and *how* entities are created through operational sequences. While traditional databases ensure that a recording session references a valid mouse (referential integrity), workflow integrity also ensures that the mouse must be created *before* the recording session can be created—preserving the temporal and operational order of your enterprise processes.
155+
156+
Workflow integrity maintains valid operation sequences through:
157+
- **Workflow dependencies** that extend foreign keys with operational semantics—parent entities must exist before child entities can be created
158+
- **Directed Acyclic Graph (DAG) structure** that prevents circular dependencies and ensures workflows can always execute
159+
- **Enforced temporal order** that guarantees operations occur in sequences that reflect real-world processes
160+
- **Computational validity** that ensures downstream results remain consistent with upstream inputs throughout the workflow
161+
162+
**Example:** An analysis pipeline cannot compute results before acquiring raw data. If `NeuronAnalysis` depends on `SpikeData`, which depends on `RecordingSession`, the database enforces that recordings are created before spike detection, which occurs before analysis—maintaining the integrity of the entire scientific workflow.
163+
164+
**Covered in:**
165+
- [Relational Workflows](../concepts/04-workflows.md) — The Relational Workflow Model and workflow dependencies
166+
- [Foreign Keys](030-foreign-keys.ipynb) — How foreign keys encode workflow dependencies
167+
- [Computation](../operations/010-computation.ipynb) — Automatic workflow execution and dependency resolution
168+
169+
---
170+
150171
# The Power of Declarative Constraints
151172

152173
Unlike application-level validation (checking rules in Python code), database constraints are:
@@ -175,7 +196,8 @@ if mouse_id not in existing_mice:
175196
DataJoint builds on SQL's integrity mechanisms with additional features:
176197

177198
- **Automatic foreign keys** from table dependencies
178-
- **Cascading deletes** that respect data pipelines
199+
- **Workflow integrity** through DAG-enforced operation sequences
200+
- **Cascading deletes** that respect data pipelines and maintain computational validity
179201
- **Transaction management** for atomic operations
180202
- **Schema validation** catching errors before database creation
181203
- **Entity relationships** expressed in intuitive Python syntax

0 commit comments

Comments
 (0)