Skip to content

Commit c933864

Browse files
authored
Add long-distance example to doc (#1361)
* Add long-distance example to doc * Update changelog
1 parent 77d2cc4 commit c933864

File tree

2 files changed

+78
-0
lines changed

2 files changed

+78
-0
lines changed

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,7 @@ ImportedLFP().drop()
9393

9494
- Add documentation for custom pipeline #1281
9595
- Add developer note on initializing `hatch` #1281
96+
- Add concrete example for long-distance restrictions #1361
9697

9798
### Pipelines
9899

docs/src/Features/Mixin.md

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,83 @@ my_table << upstream_restriction >> downstream_restriction
103103
When providing a restriction of the parent, use 'up' direction. When providing a
104104
restriction of the child, use 'down' direction.
105105

106+
For a more concrete example, imagine that we want to know which entries in the
107+
`BurstPair` table have a `1` in the `session_id`. The DataJoint-native way to
108+
approach this is to manually figure out the path from `Session` -> `Raw` ... ->
109+
`BurstPair` by looking at the diagram, or using various graph-structure
110+
methods: `parents`, `children`, `ancestors`, and `descendants`. We then use the
111+
`*` join operator for each link in the chain. Because there are some clashing
112+
secondary keys, we would need to drop them with `proj()`.
113+
114+
```python
115+
from spyglass.common import Raw, Session
116+
from spyglass.spikesorting.v1 import (
117+
BurstPair,
118+
BurstPairSelection,
119+
CurationV1,
120+
MetricCuration,
121+
MetricCurationSelection,
122+
SpikeSorting,
123+
SpikeSortingRecording,
124+
SpikeSortingRecordingSelection,
125+
SpikeSortingSelection,
126+
)
127+
128+
join = (
129+
(
130+
(
131+
(Session * Raw).proj()
132+
* SpikeSortingRecordingSelection
133+
* SpikeSortingRecording
134+
).proj()
135+
* SpikeSortingSelection
136+
* SpikeSorting
137+
).proj()
138+
* CurationV1
139+
* MetricCurationSelection
140+
).proj() * (
141+
MetricCuration * BurstPairSelection * BurstPair
142+
) * Session & "session_id LIKE '%1%'" # Last join adds back secondary keys
143+
```
144+
145+
- **Pros:**
146+
- Reliable - will not break if new tables are added elsewhere
147+
- Fast - deterministic computational process, no guess-and-check
148+
- All primary keys present, so it's easier to fetch other attributes
149+
- **Cons:**
150+
- Effortful - for long pipelines, takes time to find the path and then add
151+
relevant parentheses and `proj`
152+
- Inconvenient for one-off queries
153+
154+
Instead, we can use the long-distance operator, or the equivalent `restrict_by`
155+
method to restrict the current table. By default, `verbose` and `return_graph`
156+
are false. When turned on, these display the search process and return the
157+
`RestrGraph` object, respectively. The graph object can be used to look at
158+
the path or check the same restriction at midpoints.
159+
160+
```python
161+
from spyglass.spikesorting.v1 import BurstPair
162+
163+
burst_tbl = BurstPair()
164+
by_operator = burst_tbl << "session_id LIKE '%1%'"
165+
restr_graph = burst_tbl.restrict_by(
166+
"session_id LIKE '%1%'", direction="up", verbose=True, return_graph=True
167+
)
168+
restr_graph.path # see the list of tables in the path
169+
restr_graph.all_ft # see each table as restricted by session_restr
170+
```
171+
172+
- **Pros:**
173+
- Easy to write
174+
- Easier to read - the result is a restriction on the current table, not
175+
including any additional primary keys.
176+
- **Cons:**
177+
- Slow - ~10x processing time to run restriction compatibility checks
178+
- Unreliable - may not traverse multi-directional paths (i.e., parent ->
179+
child -> other-parent)
180+
- False negatives - may show no results if the discovered path does not
181+
follow the data provenance
182+
106183
## Delete Permission Checks
107184

108185
By default, DataJoint is unable to set delete permissions on a per-table basis.

0 commit comments

Comments
 (0)