harmonize Foreign Key narrative with Schema-as-a-Workflow paradigm

Author: dimitri-yatsenko

book/30-database-design/030-foreign-keys.ipynb

@@ -21,6 +21,17 @@
     "\n",
     "A foreign key is a column (or set of columns) in the child table that refers to the primary key of the parent table. In DataJoint, a foreign key *always* references a parent's primary key, which is a highly recommended practice for clarity and consistency.\n",
     "\n",
+    "## Foreign Keys in DataJoint: Referential Integrity + Workflow Dependencies\n",
+    "\n",
+    "In DataJoint, foreign keys serve a **dual role** that extends beyond traditional relational databases:\n",
+    "\n",
+    "1. **Referential integrity** (like traditional databases): Ensures that child references must exist in the parent table\n",
+    "2. **Workflow dependencies** (DataJoint addition): Prescribes the order of operations—the parent must be created before the child\n",
+    "\n",
+    "This transforms the schema into a **directed acyclic graph (DAG)** representing valid workflow execution sequences. The foreign key `-> Title` in `Employee` not only ensures that each employee has a valid title, but also establishes that titles must be created before employees can be assigned to them.\n",
+    "\n",
+    "For more on how DataJoint extends foreign keys with workflow semantics, see [Relational Workflows](../20-concepts/04-workflows.md).\n",
+    "\n",
     "In the following example, we define the parent table `Title` and the child table `Employee`, which references `Title`."
    ]
   },
@@ -77,7 +88,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Here the arrow `-> Title` creates a foreign key from `Employee` (child) to `Title` (parent).\n",
+    "Here the arrow `-> Title` creates a foreign key from `Employee` (child) to `Title` (parent). This foreign key:\n",
+    "- **Enforces referential integrity**: Ensures each employee has a valid title that exists in the `Title` table\n",
+    "- **Establishes workflow dependency**: Requires that titles must be created before employees can be assigned to them\n",
     "\n",
     "We can use the `dj.Diagram` class to visualize the relationships created by the foreign keys."
    ]
@@ -136,7 +149,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "The parent table `Title` is above and the child table `Employee` is below."
+    "The parent table `Title` is above and the child table `Employee` is below. The arrow direction indicates both the referential relationship (Employee references Title) and the workflow dependency (Title must be created before Employee)."
    ]
   },
   {
@@ -193,14 +206,14 @@
    "source": [
     "## The Five Effects of a Foreign Key\n",
     "\n",
-    "Foreign keys enforce **referential integrity** by regulating the relationships between a **parent table** (referenced entity set) and a **child table** (dependent entity set). In addition to defining how entities relate, foreign keys also impose important constraints on data operations. \n",
+    "Foreign keys enforce **referential integrity** by regulating the relationships between a **parent table** (referenced entity set) and a **child table** (dependent entity set). In DataJoint, they also establish **workflow dependencies** that prescribe the order of operations. In addition to defining how entities relate, foreign keys also impose important constraints on data operations. \n",
     "\n",
     "Below are the five key effects of foreign keys:\n",
     "\n",
     "### Effect 1. The primary key columns from the parent become embedded as foreign key columns in the child  \n",
     "When a foreign key relationship is established, the **primary key** (or unique key) of the parent table becomes part of the child table’s schema. The child table includes the foreign key attribute(s) with **matching name and datatype** to ensure that each row in the child table refers to a valid parent record.\n",
     "\n",
-    "If you examine the heading of `Employee`, you will find that it now contains a `title_code` field. It will have the same data type as the \n"
+    "If you examine the heading of `Employee`, you will find that it now contains a `title_code` field. It will have the same data type as the corresponding field in `Title`. \n"
    ]
   },
   {
@@ -309,7 +322,9 @@
     "\n",
     "A foreign key ensures that no \"orphaned\" records are created. An insert into the child table is only permitted if the foreign key value corresponds to an existing primary key in the parent table.\n",
     "\n",
-    "The rule is simple: **Inserts are restricted in the child, not the parent.** You can always add new job titles, but you cannot add an employee with a `title_code` that doesn't exist in the `Title` table."
+    "The rule is simple: **Inserts are restricted in the child, not the parent.** You can always add new job titles, but you cannot add an employee with a `title_code` that doesn't exist in the `Title` table.\n",
+    "\n",
+    "**In DataJoint, this enforces workflow order**: The parent entity must be created before the child entity can reference it. This ensures workflows execute in the correct sequence."
    ]
   },
   {
@@ -351,7 +366,9 @@
     "\n",
     "The rule is the inverse of the insert rule: **Deletes are restricted in the parent, not the child.** You can always delete an employee, but you cannot delete a title if it is still assigned to an employee.\n",
     "\n",
-    "In standard SQL, this operation would fail with a constraint error. DataJoint, however, implements a **cascading delete**. It will warn you that deleting the parent record will also delete all dependent child records, which can cascade through many levels of a deep hierarchy."
+    "In standard SQL, this operation would fail with a constraint error. DataJoint, however, implements a **cascading delete**. It will warn you that deleting the parent record will also delete all dependent child records, which can cascade through many levels of a deep hierarchy.\n",
+    "\n",
+    "**In DataJoint, this maintains workflow consistency**: When you delete a parent entity, all downstream workflow artifacts that depend on it are also deleted. This ensures computational validity—if the inputs are gone, any results based on those inputs must be removed as well. This is essential for maintaining workflow integrity in computational pipelines."
    ]
   },
   {
@@ -393,7 +410,9 @@
     "\n",
     "In general relational theory, databases can be configured to handle this with **cascading updates**, where changing a parent's primary key automatically propagates that change to all child records.\n",
     "\n",
-    "However, DataJoint does not support updating primary key values, as this can risk breaking referential integrity in complex scientific workflows. The preferred and safer pattern in DataJoint is to **delete the old record and insert a new one** with the updated information."
+    "However, DataJoint does not support updating primary key values, as this can risk breaking referential integrity in complex scientific workflows. The preferred and safer pattern in DataJoint is to **delete the old record and insert a new one** with the updated information.\n",
+    "\n",
+    "**In DataJoint, this preserves workflow immutability**: Workflow artifacts are treated as immutable once created. If upstream data changes, the workflow must be re-executed from that point forward. This ensures that all downstream results remain consistent with their inputs, maintaining computational validity throughout the workflow."
    ]
   },
   {
@@ -469,13 +488,17 @@
    "source": [
     "# Summary\n",
     "\n",
-    "Foreign keys ensure referential integrity by linking a child table to a parent table. This link imposes five key effects:\n",
+    "Foreign keys ensure referential integrity by linking a child table to a parent table. In DataJoint, they also establish **workflow dependencies** that prescribe the order of operations. This link imposes five key effects:\n",
     "\n",
     "1.  **Schema Embedding**: The parent's primary key is added as columns to the child table.\n",
-    "2.  **Insert Restriction**: A row cannot be added to the **child** if its foreign key doesn't match a primary key in the **parent**.\n",
-    "3.  **Delete Restriction**: A row cannot be deleted from the **parent** if it is still referenced by any rows in the **child**.\n",
-    "4.  **Update Restriction**: Updates to the primary and foreign keys are restricted to prevent inconsistencies.\n",
-    "5.  **Performance Optimization**: An index is automatically created on the foreign key in the child table to speed up searches and joins."
+    "2.  **Insert Restriction**: A row cannot be added to the **child** if its foreign key doesn't match a primary key in the **parent**. In DataJoint, this enforces workflow order—the parent must be created before the child.\n",
+    "3.  **Delete Restriction**: A row cannot be deleted from the **parent** if it is still referenced by any rows in the **child**. In DataJoint, cascading deletes maintain workflow consistency by removing dependent downstream artifacts.\n",
+    "4.  **Update Restriction**: Updates to the primary and foreign keys are restricted to prevent inconsistencies. In DataJoint, this preserves workflow immutability—workflow artifacts must be re-executed rather than updated.\n",
+    "5.  **Performance Optimization**: An index is automatically created on the foreign key in the child table to speed up searches and joins.\n",
+    "\n",
+    "**In DataJoint, foreign keys transform the schema into a directed acyclic graph (DAG)** that represents valid workflow execution sequences. The schema becomes an executable specification of your workflow, where foreign keys not only enforce referential integrity but also prescribe the order of operations and maintain computational validity throughout the workflow.\n",
+    "\n",
+    "For more on how DataJoint extends foreign keys with workflow semantics, see [Relational Workflows](../20-concepts/04-workflows.md)."
    ]
   }
  ],