highlight the role of foreign keys in defining workflows

Author: dimitri-yatsenko

book/30-database-design/030-foreign-keys.ipynb

@@ -21,12 +21,12 @@
     "\n",
     "A foreign key is a column (or set of columns) in the child table that refers to the primary key of the parent table. In DataJoint, a foreign key *always* references a parent's primary key, which is a highly recommended practice for clarity and consistency.\n",
     "\n",
-    "## Foreign Keys in DataJoint: Referential Integrity + Workflow Dependencies\n",
+    "## Referential Integrity + Workflow Dependencies\n",
     "\n",
     "In DataJoint, foreign keys serve a **dual role** that extends beyond traditional relational databases:\n",
     "\n",
     "1. **Referential integrity** (like traditional databases): Ensures that child references must exist in the parent table\n",
-    "2. **Workflow dependencies** (DataJoint addition): Prescribes the order of operations—the parent must be created before the child\n",
+    "2. **Workflow dependencies** (DataJoint's addition): Prescribes the order of operations—the parent must be created before the child\n",
     "\n",
     "This transforms the schema into a **directed acyclic graph (DAG)** representing valid workflow execution sequences. The foreign key `-> Title` in `Employee` not only ensures that each employee has a valid title, but also establishes that titles must be created before employees can be assigned to them.\n",
     "\n",
@@ -415,6 +415,27 @@
     "**In DataJoint, this preserves workflow immutability**: Workflow artifacts are treated as immutable once created. If upstream data changes, the workflow must be re-executed from that point forward. This ensures that all downstream results remain consistent with their inputs, maintaining computational validity throughout the workflow."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Effect 5: Performance Optimization with Secondary Indexes\n",
+    "\n",
+    "A secondary index is automatically created on the foreign key in the child table to accelerate common operations and queries associated with foreign keys.\n",
+    "\n",
+    "**Why indexes matter:**\n",
+    "\n",
+    "1. **Delete operations**: When deleting from the parent table, the database must look up all matching child records to enforce referential integrity. An index on the foreign key makes these lookups fast, even when dealing with large child tables.\n",
+    "\n",
+    "2. **Join operations**: When joining a parent table with a child table, the database matches the foreign key in the child to the primary key in the parent. An index on the foreign key allows the database to quickly locate matching rows, dramatically improving join performance.\n",
+    "\n",
+    "3. **Subqueries**: When checking whether a foreign key value exists in the parent table, the database uses the index to quickly verify the existence of the referenced record. This is especially important for insert operations that must validate foreign key constraints.\n",
+    "\n",
+    "**In DataJoint, this optimization is automatic**: When you define a foreign key with `-> Parent`, DataJoint automatically creates the necessary index. This ensures that workflow operations—from populating tables to cascading deletes—remain efficient even as your data grows.\n",
+    "\n",
+    "The index is created automatically by the database system, so you don't need to explicitly define it. However, understanding its existence helps you appreciate why foreign key operations remain performant even with large datasets.\n"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},