Skip to content

Commit 0b821b2

Browse files
Distinguish the roles of Manual vs Lookup tables (#200)
* docs: distinguish the roles of Manual vs Lookup tables Both tiers hold entered (not computed) rows; the distinction is where the rows come from. Make that explicit across the docs: - explanation/relational-workflow-model.md: new "Manual vs. Lookup" subsection and sharper tier-table roles — Manual rows arrive at runtime from outside the pipeline; Lookup rows are part of the schema definition (declared via `contents`, versioned with the code). Includes the "where does a row come from?" test and the common misclassification (using a Lookup for runtime-entered data belongs in Manual). - how-to/define-tables.md: a practical "Manual or Lookup?" note by the Lookup-contents example, plus clearer tier-table purposes. - reference/specs/table-declaration.md: sharpen the tier "Purpose" cells. * docs: note that Lookup content updates flow through CI/CD Because Lookup contents live in the schema code, changing them is a code change (edit contents -> review -> deploy) rather than a runtime insert — reinforcing the Manual (runtime) vs Lookup (code/CI-CD) distinction.
1 parent 9738acb commit 0b821b2

3 files changed

Lines changed: 56 additions & 6 deletions

File tree

src/explanation/relational-workflow-model.md

Lines changed: 33 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -134,15 +134,46 @@ Tables are classified into tiers by data-entry mode:
134134

135135
| Tier | Role | `make()` |
136136
|------|------|----------|
137-
| **Manual** | Receive direct user entry | No |
138-
| **Lookup** | Hold reference data | No |
137+
| **Manual** | Rows entered at runtime from outside the pipeline (people, forms, instruments, imports) | No |
138+
| **Lookup** | Reference rows defined in the schema itself via `contents` | No |
139139
| **Imported** | Reach out to data sources outside DataJoint (instruments, ELNs, external databases) | Yes |
140140
| **Computed** | Derive their contents entirely from upstream DataJoint tables | Yes |
141141

142142
Imported and Computed tables define computations via `make()` methods. The
143143
`make()` method specifies how each entity is derived — declared within the
144144
table definition, not in an external workflow file.
145145

146+
#### Manual vs. Lookup
147+
148+
Manual and Lookup tables are both **entry points** — their rows are entered
149+
rather than derived by a `make()` — but they differ in *where the rows come
150+
from*:
151+
152+
- A **Manual** table's rows arrive at **runtime**, from outside the pipeline: a
153+
person typing into a form, a LIMS, an instrument, or an import from another
154+
system. Its contents are specific to a particular project or experiment and
155+
differ from one deployment to the next. Manual tables are the pipeline's origin
156+
points — e.g. `Mouse`, `Session`, `Scan`.
157+
- A **Lookup** table's rows are **part of the schema definition**, declared in
158+
code through the `contents` attribute and versioned alongside the table. Its
159+
contents are the same wherever the schema is deployed and change only when the
160+
code changes. Use it for reference values that belong to the pipeline's design:
161+
parameter sets, method definitions, controlled vocabularies, enumerations —
162+
e.g. `SegmentationParam`.
163+
164+
The quick test is *where does a row come from?* If it is fixed in the committed
165+
schema (`contents`), it is a **Lookup**; if it arrives at runtime, it is a
166+
**Manual** table. A common mistake is to use a Lookup for data that is actually
167+
entered at runtime (for example, filled in through a dashboard form). If a
168+
table's rows do not come from its committed `contents`, it belongs in the
169+
**Manual** tier.
170+
171+
Because Lookup content lives in the code, **changing it is a code change**:
172+
you edit `contents` and redeploy, so updates flow through the same
173+
review-and-deploy (CI/CD) process as any other schema change — versioned and
174+
reproducible across deployments. Manual content, by contrast, is entered at
175+
runtime and never touches the codebase.
176+
146177
### Master-part relationships
147178

148179
Master-part relationships declare transactional grouping directly in the

src/how-to/define-tables.md

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,8 +30,8 @@ class MyTable(dj.Manual):
3030

3131
| Type | Base Class | Purpose |
3232
|------|------------|---------|
33-
| Manual | `dj.Manual` | User-entered data |
34-
| Lookup | `dj.Lookup` | Reference data with `contents` |
33+
| Manual | `dj.Manual` | Data entered at runtime (users, forms, instruments, imports) |
34+
| Lookup | `dj.Lookup` | Reference data defined in the schema via `contents` |
3535
| Imported | `dj.Imported` | Data from external sources |
3636
| Computed | `dj.Computed` | Derived data |
3737
| Part | `dj.Part` | Child of master table |
@@ -251,6 +251,25 @@ class TaskType(dj.Lookup):
251251
]
252252
```
253253

254+
### Manual or Lookup?
255+
256+
Both tiers hold rows that are *entered* rather than computed, so the question is
257+
**where a row comes from**:
258+
259+
- Use **`dj.Lookup`** when the rows are part of the schema's design and belong in
260+
the code — parameter sets, method definitions, controlled vocabularies,
261+
enumerations. They are declared in `contents`, versioned with the table, and
262+
identical in every deployment until the code changes. Updating a Lookup means
263+
editing `contents` and redeploying — the change flows through your normal CI/CD
264+
process, not a runtime insert.
265+
- Use **`dj.Manual`** when the rows are entered at runtime and are specific to a
266+
project or experiment — subjects, sessions, samples, or anything typed into a
267+
form, ingested from a file, or read from an instrument.
268+
269+
If you find yourself populating a "Lookup" table at runtime (for example, from a
270+
dashboard form) rather than from its committed `contents`, make it `dj.Manual`
271+
instead — its rows are runtime data, not part of the schema definition.
272+
254273
## Part Tables
255274

256275
```python

src/reference/specs/table-declaration.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,8 @@ class TableName(dj.Manual):
2323

2424
| Tier | Base Class | Table Prefix | Purpose |
2525
|------|------------|--------------|---------|
26-
| Manual | `dj.Manual` | (none) | User-entered data |
27-
| Lookup | `dj.Lookup` | `#` | Reference/enumeration data |
26+
| Manual | `dj.Manual` | (none) | Data entered at runtime (users, instruments, imports) |
27+
| Lookup | `dj.Lookup` | `#` | Reference data defined in the schema via `contents` |
2828
| Imported | `dj.Imported` | `_` | Data from external sources |
2929
| Computed | `dj.Computed` | `__` | Derived from other tables |
3030
| Part | `dj.Part` | `master__` | Detail records of master table |

0 commit comments

Comments
 (0)