[ENH] BEP032: Move anatomical location to the electrode level

src/modality-specific-files/microelectrode-electrophysiology.md

@@ -326,26 +326,96 @@ This rule is different from the electrodes.tsv table of the [iEEG modality](intr
 
 To specify electrode positions in surgical space, individual anatomical space, or a common coordinate system (such as the Allen CCF), use an additional `*_electrodes.tsv` file with a [`space-<label>`](../appendices/entities.md#space) entity. See the [`*_coordsystem.json` section](#coordinate-system-json-_coordsystemjson) for details on defining these coordinate systems.
 
+### Anatomical Location
+
+Anatomical location is described at two levels of detail.
+The `anatomical_location` column of `*_probes.tsv` names the structure in which the probe as a whole
+is placed, which for a probe spanning several structures MUST be a structure containing all of them,
+and may be as coarse as `brain` or a single hemisphere.
+The `anatomical_location` column of `*_electrodes.tsv` names the structure in which an individual
+recording site is located, and is where a per-contact localization belongs.
+Only the electrode-level column can describe a probe that passes through several structures,
+which is common for long shank probes.
+
+Terms may be taken from a species-independent ontology such as [Uberon](https://obophenotype.github.io/uberon/) or from a species-specific
+atlas or ontology such as the [Mouse Brain Atlas Ontology](https://www.ebi.ac.uk/ols4/ontologies/mba), and a species-specific is often the better choice.
+The source the terms come from SHOULD be documented in the sidecar file, in the way BIDS
+documents any other tabular column, as described in
+[Tabular files](../common-principles.md#tabular-files).
+Where the individual terms resolve, give each one its own `TermURL` under `Levels`.
+Where they do not, name the atlas in the `ReferenceAtlas` field of the sidecar.
+
+The method used to determine the location SHOULD be recorded.
+Where it differs between recording sites, use the `localization_method` column of
+`*_electrodes.tsv`.
+Where the same method applies to every electrode in the file, it MAY be given once in the
+`LocalizationMethod` field of the corresponding `*_electrodes.json` file instead,
+but it MUST NOT be given in both places.
+
+{{ MACROS___make_sidecar_table("microephys.microephysElectrodeLocalization") }}
+
+In the following `*_electrodes.json` example the atlas resolves its individual structures,
+so each term used in the table is given its own `TermURL`:
+
+```JSON
+{
+  "anatomical_location": {
+    "Description": "Structure the electrode is located in, from the Allen Mouse Brain Atlas",
+    "Levels": {
+      "MOp": {
+        "Description": "Primary motor area",
+        "TermURL": "https://atlas.brain-map.org/atlas?atlas=602630314#structure=985"
+      },
+      "CA1": {
+        "Description": "Field CA1",
+        "TermURL": "https://atlas.brain-map.org/atlas?atlas=602630314#structure=382"
+      }
+    }
+  },
+  "LocalizationMethod": "histology",
+  "ReferenceAtlas": "Allen Mouse Brain Common Coordinate Framework v3"
+}
+```
+
+In the next example the atlas publishes region names but nothing to resolve them to,
+so there are no `TermURL` values to give and `ReferenceAtlas` is what makes the names
+in `anatomical_location` interpretable:
+
+```JSON
+{
+  "LocalizationMethod": "stereotaxic coordinates",
+  "ReferenceAtlas": "D99 macaque atlas v2.0"
+}
+```
+
+In both examples the localization method is given once in the sidecar because it is the same
+for every electrode in the file.
+
+Because a dataset may contain more than one `*_electrodes.tsv` file for the same recording,
+distinguished by the [`space-<label>`](../appendices/entities.md#space) entity,
+localizations produced by different atlases or different methods can be provided alongside one another,
+each with its own `ReferenceAtlas` and `LocalizationMethod`.
+
 ### Example `*_electrodes.tsv`
 
 **Extracellular electrophysiology example (probe-relative coordinates):**
 
 ```tsv
-name	probe_name	hemisphere	x	y	z	impedance	shank_id	size	material	location
-e001	probe01	L	0	0	0	1.2	0	15	iridium-oxide	MOp
-e002	probe01	L	0	0	25	1.1	0	15	iridium-oxide	MOp
-e003	probe01	L	0	0	50	1.3	0	15	iridium-oxide	MOp
-e004	probe01	L	0	0	75	1.4	0	15	iridium-oxide	MOp
-e005	probe02	R	0	0	0	2.1	n/a	12	tungsten	CA1
-e006	probe02	R	0	0	15	2.3	n/a	12	tungsten	CA1
-e007	probe02	R	0	0	30	1.9	n/a	12	tungsten	CA1
-e008	probe02	R	0	0	45	2.0	n/a	12	tungsten	CA1
+name	probe_name	hemisphere	x	y	z	impedance	shank_id	size	material	anatomical_location	localization_method
+e001	probe01	L	0	0	0	1.2	0	15	iridium-oxide	MOp	histology
+e002	probe01	L	0	0	25	1.1	0	15	iridium-oxide	MOp	histology
+e003	probe01	L	0	0	50	1.3	0	15	iridium-oxide	MOp	histology
+e004	probe01	L	0	0	75	1.4	0	15	iridium-oxide	MOp	histology
+e005	probe02	R	0	0	0	2.1	n/a	12	tungsten	CA1	histology
+e006	probe02	R	0	0	15	2.3	n/a	12	tungsten	CA1	histology
+e007	probe02	R	0	0	30	1.9	n/a	12	tungsten	CA1	histology
+e008	probe02	R	0	0	45	2.0	n/a	12	tungsten	CA1	histology
 ```
 
 **Intracellular electrophysiology example:**
 
 ```tsv
-name	probe_name	hemisphere	x	y	z	impedance	pipette_solution	internal_pipette_diameter	external_pipette_diameter	material	location
+name	probe_name	hemisphere	x	y	z	impedance	pipette_solution	internal_pipette_diameter	external_pipette_diameter	material	anatomical_location
 patch01	pipette01	L	0	0	0	5.2	K-gluconate	1.5	2.5	borosilicate-glass	VISp2/3
 patch02	pipette02	R	0	0	0	4.8	K-gluconate	1.5	2.5	borosilicate-glass	VISp2/3
 sharp01	pipette03	L	0	0	0	80	3M KCl	0.5	1.0	borosilicate-glass	PL5
@@ -365,18 +435,18 @@ This file contains the probe ID, the type of recording (acute/chronic), and the
 **Extracellular electrophysiology example:**
 
 ```tsv
-probe_name	type	AP	ML	DV	AP_angle	ML_angle	rotation_angle	hemisphere	manufacturer	device_serial_number	electrode_count	width	height	depth	coordinate_reference_point	anatomical_reference_point	associated_brain_region	associated_brain_region_id	reference_atlas	material
-probe01	silicon-probe	-2.5	1.5	-4.0	15	0	0	L	IMEC	NP1100-2205	384	70	20	10	tip	Bregma	Primary Motor Cortex	MOp	Franklin-Paxinos	silicon
-probe02	tetrode	-1.2	-2.1	-3.5	0	10	45	R	Neuralynx	TT-12345	4	n/a	n/a	n/a	tip	Bregma	Hippocampus CA1	CA1	Paxinos-Watson	tungsten
+probe_name	type	AP	ML	DV	AP_angle	ML_angle	rotation_angle	hemisphere	manufacturer	device_serial_number	electrode_count	width	height	depth	coordinate_reference_point	anatomical_reference_point	anatomical_location	material
+probe01	silicon-probe	-2.5	1.5	-4.0	15	0	0	L	IMEC	NP1100-2205	384	70	20	10	tip	Bregma	isocortex	silicon
+probe02	tetrode	-1.2	-2.1	-3.5	0	10	45	R	Neuralynx	TT-12345	4	n/a	n/a	n/a	tip	Bregma	CA1	tungsten
 ```
 
 **Intracellular electrophysiology example:**
 
 ```tsv
-probe_name	type	AP	ML	DV	AP_angle	ML_angle	rotation_angle	hemisphere	manufacturer	electrode_count	coordinate_reference_point	anatomical_reference_point	associated_brain_region	associated_brain_region_id	reference_atlas
-pipette01	patch-pipette	-1.8	0.5	-2.2	30	0	0	L	Sutter	1	tip	Bregma	Visual Cortex Layer 2/3	VISp2/3	AllenCCFv3
-pipette02	patch-pipette	-1.8	-0.5	-2.2	30	0	0	R	Sutter	1	tip	Bregma	Visual Cortex Layer 2/3	VISp2/3	AllenCCFv3
-pipette03	sharp-electrode	-3.2	1.2	-3.8	20	5	0	L	WPI	1	tip	Bregma	Prefrontal Cortex Layer 5	PL5	Franklin-Paxinos
+probe_name	type	AP	ML	DV	AP_angle	ML_angle	rotation_angle	hemisphere	manufacturer	electrode_count	coordinate_reference_point	anatomical_reference_point	anatomical_location
+pipette01	patch-pipette	-1.8	0.5	-2.2	30	0	0	L	Sutter	1	tip	Bregma	VISp2/3
+pipette02	patch-pipette	-1.8	-0.5	-2.2	30	0	0	R	Sutter	1	tip	Bregma	VISp2/3
+pipette03	sharp-electrode	-3.2	1.2	-3.8	20	5	0	L	WPI	1	tip	Bregma	PL5
 ```
 
 For details on the surgical coordinate system used to describe probe placement during surgery (AP, ML, DV, angles, and

src/schema/objects/columns.yaml

@@ -78,26 +78,36 @@ AP__probes:
     Positive values are anterior to the reference point.
   type: number
   unit: mm
-associated_brain_region:
-  name: associated_brain_region
-  display_name: Associated brain region
-  description: |
-    A textual indication on the location of the probe,
-    preferably species-independent terms as obtained, for example from Uberon.
-  type: string
-associated_brain_region_id:
-  name: associated_brain_region_id
-  display_name: Associated brain region identifier
+anatomical_location:
+  name: anatomical_location
+  display_name: Anatomical location
   description: |
-    An identifier of the associated brain region based on the Uberon ontology
-    for anatomical structures in animals, for example "UBERON:0010415"
-  type: string
-associated_brain_region_quality_type:
-  name: associated_brain_region_quality_type
-  display_name: Associated brain region quality type
+    The anatomical structure in which the electrode is located
+    (for example, `cortical layer 3`, `CA1`, `MOp`).
+    Terms from a species-specific atlas are acceptable, and are often preferable to
+    species-independent terms.
+    The ontology or atlas the terms are taken from SHOULD be documented in the
+    corresponding `*_electrodes.json` file, using `TermURL` for the column and,
+    where the individual terms are resolvable, `Levels` to map each term to its own
+    `TermURL` (see [Tabular files](SPEC_ROOT/common-principles.md#tabular-files)).
+  type: string
+# anatomical_location for probes.tsv files, which describes the probe as a whole and so is
+# coarser than its electrodes.tsv counterpart
+anatomical_location__probes:
+  name: anatomical_location
+  display_name: Anatomical location
   description: |
-    The method used to identify the associated brain region (estimated|proof)
-    depending on anatomical pictures proofing the location or indirect estimation of the location.
+    The anatomical structure in which the probe is placed,
+    for example, `brain`, `left hemisphere`, `isocortex`, or `CA1`.
+    This column describes the probe as a whole, so the structure named here SHOULD be one
+    that contains every recording site on the probe.
+    A probe that spans several structures SHOULD therefore be described here by a
+    containing structure, with the location of each recording site given in the
+    `anatomical_location` column of the `*_electrodes.tsv` file.
+    Terms from a species-specific atlas are acceptable, and are often preferable to
+    species-independent terms, and the source they come from SHOULD be documented in the
+    corresponding `*_probes.json` file as described for the `anatomical_location` column of
+    `*_electrodes.tsv`.
   type: string
 cardiac:
   name: cardiac
@@ -443,11 +453,16 @@ index:
   description: |
     The label integer index.
   type: integer
-location:
-  name: location
-  display_name: Location
+localization_method:
+  name: localization_method
+  display_name: Localization method
   description: |
-    An indication on the location of the electrode (for example, `cortical layer 3`, `CA1`).
+    The method used to determine the anatomical location of the electrode,
+    for example, `histology`, `atlas registration`, `stereotaxic coordinates`,
+    or `post-operative imaging`.
+    Where the same method applies to every electrode in the file, it MAY instead be given
+    once in the `LocalizationMethod` field of the corresponding `*_electrodes.json` file.
+    It MUST NOT be given in both places.
   type: string
 interelectrode_distance:
   name: interelectrode_distance
@@ -731,13 +746,6 @@ reference__ieeg:
     Specification of the reference (for example, `mastoid`, `ElectrodeName01`, `intracranial`, `CAR`, `other`, `n/a`).
     If the channel is not an electrode channel (for example, a microphone channel) use `n/a`.
   type: string
-reference_atlas:
-  name: reference_atlas
-  display_name: Reference atlas
-  description: |
-    Name of reference atlas used for associated brain region identification,
-    preferably an [EBRAINS-supported atlas](https://www.ebrains.eu/brain-atlases/reference-atlases/#services).
-  type: string
 reference_frame:
   name: reference_frame
   display_name: Reference frame

src/schema/objects/metadata.yaml

@@ -2293,6 +2293,19 @@ License:
     The corresponding full license text MAY be specified in an additional
     `LICENSE` file.
   type: string
+LocalizationMethod:
+  name: LocalizationMethod
+  display_name: Localization Method
+  description: |
+    The method used to determine the anatomical location of the electrodes described in the
+    corresponding `*_electrodes.tsv` file,
+    for example, `"histology"`, `"atlas registration"`, `"stereotaxic coordinates"`,
+    or `"post-operative imaging"`.
+    This field applies to every electrode in the file.
+    Where the method differs between electrodes, use the `localization_method` column of
+    `*_electrodes.tsv` instead.
+    The method MUST NOT be specified in both places.
+  type: string
 LongName:
   name: LongName
   display_name: Long Name
@@ -3456,6 +3469,18 @@ RecordingType:
     - $ref: objects.enums.continuous.value
     - $ref: objects.enums.epoched.value
     - $ref: objects.enums.discontinuous.value
+ReferenceAtlas:
+  name: ReferenceAtlas
+  display_name: Reference Atlas
+  description: |
+    Name and, where applicable, version of the reference atlas from which the values of
+    `anatomical_location` in the corresponding `*_electrodes.tsv` file are taken,
+    for example, `"Allen Mouse Brain Common Coordinate Framework v3"` or `"D99 macaque atlas v2.0"`.
+    Some atlases publish region names but no identifiers that can be looked up, so that the
+    terms cannot be documented with a `TermURL`.
+    Where `anatomical_location` is taken from such an atlas, this field is what makes the
+    names interpretable and SHOULD be given.
+  type: string
 ReferencesAndLinks:
   name: ReferencesAndLinks
   display_name: References And Links

src/schema/rules/checks/microephys.yaml

@@ -17,6 +17,46 @@ MicroephysRequiredCoordsystemWithSpace:
   checks:
     - associations.coordsystem != null
 
+# The localization method belongs either in the electrodes table, where it can vary
+# between electrodes, or in the sidecar, where it applies to all of them, but not in both.
+MicroephysLocalizationMethodDuplicated:
+  issue:
+    code: MICROEPHYS_LOCALIZATION_METHOD_DUPLICATED
+    message: |
+      The localization method is given both in the `localization_method` column of
+      `*_electrodes.tsv` and in the `LocalizationMethod` field of the corresponding
+      `*_electrodes.json` file. It MUST be given in only one of the two.
+    level: error
+  selectors:
+    - intersects([datatype], ["ecephys", "icephys"])
+    - suffix == "electrodes"
+    - extension == ".tsv"
+    - type(columns.localization_method) != "null"
+  checks:
+    - '!("LocalizationMethod" in sidecar)'
+
+# Anatomical location terms are interpretable if the sidecar documents where they came
+# from, either by resolving the terms themselves or by naming the atlas they were taken from.
+MicroephysUndocumentedAnatomicalLocation:
+  issue:
+    code: MICROEPHYS_ANATOMICAL_LOCATION_UNDOCUMENTED
+    message: |
+      The `*_electrodes.tsv` file gives anatomical locations, but the corresponding
+      `*_electrodes.json` file neither documents the terms with `TermURL` or `Levels`
+      nor names the atlas they were taken from in `ReferenceAtlas`,
+      so the terms cannot be interpreted.
+    level: warning
+  selectors:
+    - intersects([datatype], ["ecephys", "icephys"])
+    - suffix == "electrodes"
+    - extension == ".tsv"
+    - columns.anatomical_location != null
+  checks:
+    - |
+      sidecar.anatomical_location.TermURL != null ||
+      sidecar.anatomical_location.Levels != null ||
+      sidecar.ReferenceAtlas != null
+
 # Stereotaxic coordinates cannot be interpreted without knowing the point they are
 # measured from, so requiring the reference point is conditioned on their presence.
 MicroephysRequiredAnatomicalReferencePoint:

src/schema/rules/sidecars/microephys.yaml

@@ -108,3 +108,13 @@ microephysTaskInformation:
         distinguishing between eyes open and eyes closed paradigms.
     CogAtlasID: optional
     CogPOID: optional
+
+# Metadata for the electrodes table, describing how the anatomical locations in it
+# were determined and which atlas they are taken from.
+microephysElectrodeLocalization:
+  selectors:
+    - intersects([datatype], ["ecephys", "icephys"])
+    - suffix == "electrodes"
+  fields:
+    LocalizationMethod: optional
+    ReferenceAtlas: optional

src/schema/rules/tabular_data/microephys.yaml

@@ -33,10 +33,7 @@ microephysProbes:
       level: optional
       level_addendum: required if `AP`, `ML`, or `DV` are present
     hemisphere__probes: recommended
-    associated_brain_region: recommended
-    associated_brain_region_id: recommended
-    associated_brain_region_quality_type: recommended
-    reference_atlas: recommended
+    anatomical_location__probes: recommended
     material__probes: optional
   index_columns: [probe_name]
   additional_columns: allowed_if_defined
@@ -95,7 +92,8 @@ microephysElectrodes:
     size__microephys: optional
     electrode_shape: optional
     material: optional
-    location: recommended
+    anatomical_location: recommended
+    localization_method: optional
     pipette_solution: optional
     internal_pipette_diameter: optional
     external_pipette_diameter: optional