Skip to content

Commit a486cf7

Browse files
lawrence-mbflawrence-mbf
authored
Merge pull request #253 from NeurodataWithoutBorders/debug-pipeline
Fix Pipeline nwbtest errors
2 parents 7279c93 + 044979f commit a486cf7

File tree

5 files changed

+208
-7
lines changed

5 files changed

+208
-7
lines changed

‎generateCore.m‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -26,20 +26,22 @@ function generateCore(version)
2626
validateattributes(version, {'char'}, {'scalartext'});
2727
end
2828

29-
[nwbLocation, ~, ~] = fileparts(mfilename('fullpath'));
30-
schemaPath = fullfile(nwbLocation, 'nwb-schema', version);
29+
matNwbLocation = misc.getMatnwbDir();
30+
schemaPath = fullfile(matNwbLocation, 'nwb-schema', version);
3131
corePath = fullfile(schemaPath, 'core', 'nwb.namespace.yaml');
3232
commonPath = fullfile(schemaPath,...
3333
'hdmf-common-schema', ...
3434
'common',...
3535
'namespace.yaml');
3636
assert(2 == exist(corePath, 'file'),...
37-
'MATNWB:nwbread:missingSpec',...
37+
'MATNWB:GenerateCore:missingCoreSchema',...
3838
'Cannot find suitable core namespace for schema version `%s`',...
3939
version);
40+
assert(2 == exist(commonPath, 'file'),...
41+
'MatNWB:GenerateCore:MissingCoreSchema',...
42+
'Cannot find suitable hdmf-common namespace for schema version `%s`',...
43+
version);
4044

41-
if 2 == exist(commonPath, 'file')
42-
generateExtension(commonPath);
43-
end
45+
generateExtension(commonPath);
4446
generateExtension(corePath);
4547
end

‎nwb-schema/2.2.5/hdmf-common-schema‎

Lines changed: 0 additions & 1 deletion
This file was deleted.

‎nwb-schema/2.2.5/hdmf-common-schema/common/namespace.yaml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
namespaces:
2+
- name: hdmf-common
3+
doc: Common data structures provided by HDMF
4+
author:
5+
- Andrew Tritt
6+
- Oliver Ruebel
7+
- Ryan Ly
8+
- Ben Dichter
9+
contact:
10+
- [email protected]
11+
- [email protected]
12+
- [email protected]
13+
- [email protected]
14+
full_name: HDMF Common
15+
schema:
16+
- doc: data types for a column-based table
17+
source: table.yaml
18+
title: Table data types
19+
- doc: data types for different types of sparse matrices
20+
source: sparse.yaml
21+
version: 1.1.3

‎nwb-schema/2.2.5/hdmf-common-schema/common/sparse.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
groups:
2+
- data_type_def: CSRMatrix
3+
doc: a compressed sparse row matrix
4+
attributes:
5+
- name: shape
6+
dtype: int
7+
shape:
8+
- 2
9+
doc: the shape of this sparse matrix
10+
datasets:
11+
- name: indices
12+
dtype: int
13+
shape:
14+
- null
15+
doc: column indices
16+
- name: indptr
17+
dtype: int
18+
shape:
19+
- null
20+
doc: index pointer
21+
- name: data
22+
shape:
23+
- null
24+
doc: values in the matrix

‎nwb-schema/2.2.5/hdmf-common-schema/common/table.yaml‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
datasets:
2+
- data_type_def: Data
3+
doc: An abstract data type for a dataset.
4+
5+
- data_type_def: Index
6+
data_type_inc: Data
7+
doc: Pointers that index data values.
8+
attributes:
9+
- name: target
10+
dtype:
11+
target_type: Data
12+
reftype: object
13+
doc: Target dataset that this index applies to.
14+
15+
- data_type_def: VectorData
16+
data_type_inc: Data
17+
doc: An n-dimensional dataset representing a column of a DynamicTable.
18+
If used without an accompanying VectorIndex, first dimension is
19+
along the rows of the DynamicTable and each step along the first
20+
dimension is a cell of the larger table. VectorData can also be
21+
used to represent a ragged array if paired with a VectorIndex.
22+
This allows for storing arrays of varying length in a single cell
23+
of the DynamicTable by indexing into this VectorData. The first
24+
vector is at VectorData[0:VectorIndex(0)+1]. The second vector is at
25+
VectorData[VectorIndex(0)+1:VectorIndex(1)+1], and so on.
26+
dims:
27+
- - dim0
28+
- - dim0
29+
- dim1
30+
- - dim0
31+
- dim1
32+
- dim2
33+
- - dim0
34+
- dim1
35+
- dim2
36+
- dim3
37+
shape:
38+
- - null
39+
- - null
40+
- null
41+
- - null
42+
- null
43+
- null
44+
- - null
45+
- null
46+
- null
47+
- null
48+
attributes:
49+
- name: description
50+
dtype: text
51+
doc: Description of what these vectors represent.
52+
53+
- data_type_def: VectorIndex
54+
data_type_inc: Index
55+
doc: Used with VectorData to encode a ragged array. An array of indices
56+
into the first dimension of the target VectorData, and forming a map
57+
between the rows of a DynamicTable and the indices of the VectorData.
58+
dims:
59+
- num_rows
60+
shape:
61+
- null
62+
attributes:
63+
- name: target
64+
dtype:
65+
target_type: VectorData
66+
reftype: object
67+
doc: Reference to the target dataset that this index applies to.
68+
69+
- data_type_def: ElementIdentifiers
70+
data_type_inc: Data
71+
default_name: element_id
72+
dtype: int
73+
dims:
74+
- num_elements
75+
shape:
76+
- null
77+
doc: A list of unique identifiers for values within a dataset, e.g. rows of a DynamicTable.
78+
79+
- data_type_def: DynamicTableRegion
80+
data_type_inc: VectorData
81+
dtype: int
82+
doc: DynamicTableRegion provides a link from one table to an index or region of another. The `table` attribute is a
83+
link to another `DynamicTable`, indicating which table is referenced, and the data is int(s) indicating the row(s)
84+
(0-indexed) of the target array. `DynamicTableRegion`s can be used to associate rows with repeated meta-data without
85+
data duplication. They can also be used to create hierarchical relationships between multiple `DynamicTable`s.
86+
`DynamicTableRegion` objects may be paired with a `VectorIndex` object to create ragged references, so a single cell
87+
of a `DynamicTable` can reference many rows of another `DynamicTable`.
88+
dims:
89+
- num_rows
90+
shape:
91+
- null
92+
attributes:
93+
- name: table
94+
dtype:
95+
target_type: DynamicTable
96+
reftype: object
97+
doc: Reference to the DynamicTable object that this region applies to.
98+
- name: description
99+
dtype: text
100+
doc: Description of what this table region points to.
101+
102+
groups:
103+
- data_type_def: Container
104+
doc: An abstract data type for a generic container storing collections of data and
105+
metadata. Base type for all data and metadata containers.
106+
107+
- data_type_def: DynamicTable
108+
data_type_inc: Container
109+
doc: A group containing multiple datasets that are aligned on the first dimension
110+
(Currently, this requirement if left up to APIs to check and enforce). Apart from
111+
a column that contains unique identifiers for each row there are no other required
112+
datasets. Users are free to add any number of VectorData objects here. Table functionality
113+
is already supported through compound types, which is analogous to storing an
114+
array-of-structs. DynamicTable can be thought of as a struct-of-arrays. This provides
115+
an alternative structure to choose from when optimizing storage for anticipated
116+
access patterns. Additionally, this type provides a way of creating a table without
117+
having to define a compound type up front. Although this convenience may be attractive,
118+
users should think carefully about how data will be accessed. DynamicTable is
119+
more appropriate for column-centric access, whereas a dataset with a compound
120+
type would be more appropriate for row-centric access. Finally, data size should
121+
also be taken into account. For small tables, performance loss may be an acceptable
122+
trade-off for the flexibility of a DynamicTable. For example, DynamicTable was
123+
originally developed for storing trial data and spike unit metadata. Both of these
124+
use cases are expected to produce relatively small tables, so the spatial locality
125+
of multiple datasets present in a DynamicTable is not expected to have a significant
126+
performance impact. Additionally, requirements of trial and unit metadata tables
127+
are sufficiently diverse that performance implications can be overlooked in favor
128+
of usability.
129+
attributes:
130+
- name: colnames
131+
dtype: text
132+
dims:
133+
- num_columns
134+
shape:
135+
- null
136+
doc: The names of the columns in this table. This should be used to specify
137+
an order to the columns.
138+
- name: description
139+
dtype: text
140+
doc: Description of what is in this dynamic table.
141+
datasets:
142+
- name: id
143+
data_type_inc: ElementIdentifiers
144+
dtype: int
145+
dims:
146+
- num_rows
147+
shape:
148+
- null
149+
doc: Array of unique identifiers for the rows of this dynamic table.
150+
- data_type_inc: VectorData
151+
doc: Vector columns of this dynamic table.
152+
quantity: '*'
153+
- data_type_inc: VectorIndex
154+
doc: Indices for the vector columns of this dynamic table.
155+
quantity: '*'

0 commit comments

Comments
 (0)