Skip to content

Commit cf3fe73

Browse files
committed
Treenode export: add more metadata to treenode archive exports
The ZIP files generated by the treenode archive export contain now two metadata files: nodes.csv and metadata.json. The first one was called metadata.csv before. The second one is new and contains information on the project, stacks and exporting user.
1 parent 4acbd2e commit cf3fe73

File tree

3 files changed

+34
-12
lines changed

3 files changed

+34
-12
lines changed

CHANGELOG.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,11 @@ Import/Export widget:
213213
- The treenode archive creation dialog allows now also to specify the Z radius.
214214
This allows to export an image block for each treenode.
215215

216+
- The ZIP files generated by the treenode archive export contain now two
217+
metadata files: nodes.csv and metadata.json. The first one was called
218+
metadata.csv before. The second one is new and contains information on the
219+
project, stacks and exporting user.
220+
216221
Miscellaneous:
217222

218223
- The built-in API docs (/apis endpoint) now supports deep links, which allows

django/applications/catmaid/control/treenodeexport.py

Lines changed: 21 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,8 @@
1515
from catmaid.control.common import get_relation_to_id_map, id_generator, get_request_bool
1616
from catmaid.control.cropping import (collect_stack_mirros, CropJob,
1717
extract_substack, ImageRetrievalError)
18-
from catmaid.models import ClassInstanceClassInstance, TreenodeConnector, \
19-
Message, User, UserRole, Stack, Treenode
18+
from catmaid.models import (ClassInstanceClassInstance, TreenodeConnector,
19+
Message, Project, User, UserRole, Stack, Treenode)
2020

2121
from celery import shared_task
2222

@@ -196,8 +196,12 @@ def export_single_node(self, treenode) -> None:
196196

197197
def post_process(self, nodes) -> None:
198198
""" Create a meta data file for all the nodes passed (usually all of the
199-
ones queries before). This file is a table with the following columns:
199+
ones queries before). This file is called nodes.csv and is a table with
200+
the following columns:
200201
<treenode id> <parent id> <#presynaptic sites> <#postsynaptic sites> <x> <y> <z>
202+
203+
Additionally, a file metadata.json is created, which contains general
204+
information on the referenced dataset.
201205
"""
202206
# Get pre- and post synaptic sites
203207
presynaptic_to_rel = self.relation_map['presynaptic_to']
@@ -236,15 +240,26 @@ def post_process(self, nodes) -> None:
236240
ls.append(line)
237241

238242
# Save metdata for each skeleton to files
243+
project = Project.objects.get(id=self.job.project_id)
244+
image_stacks = list(map(lambda s: s.title, project.stacks.all()))
239245
for skid, metadata in skid_to_metadata.items():
240246
path = self.skid_to_neuron_folder[skid]
241-
with open(os.path.join(path, 'metadata.csv'), 'w') as f:
247+
with open(os.path.join(path, 'nodes.csv'), 'w') as f:
242248
f.write("This CSV file contains meta data for CATMAID skeleton " \
243-
"%s. The columns represent the following data:\n" % skid)
249+
f"{skid}. The columns represent the following data:\n")
244250
f.write("treenode-id, parent-id, # presynaptic sites, " \
245251
"# postsynaptic sites, x, y, z\n")
246252
for line in metadata:
247-
f.write("%s\n" % line)
253+
f.write(f"{line}\n")
254+
with open(os.path.join(path, 'metadata.json'), 'w') as f:
255+
f.write(json.dumps({
256+
'project_id': project.id,
257+
'project_name': project.title,
258+
'skeleton_id': skid,
259+
'user_id': self.job.user.id,
260+
'user_name': self.job.user.username,
261+
'image_stacks': image_stacks,
262+
}, indent=4))
248263

249264
class ConnectorExporter(TreenodeExporter):
250265
""" Most of the infrastructure can be used for both treenodes and

django/applications/catmaid/static/js/widgets/import-export-widget.js

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -73,12 +73,14 @@ the selected neurons.
7373
The generated <em>tar.gz</em> archive contains one folder for every
7474
selected neuron, named after its ID. Such a folder contains image files for
7575
every treenode of the neuron's skeleton(s), named <em>treenode-id.tiff</em>.
76-
Along those files a meta data file, named <em>metadata.csv</em>, is created.
77-
It contains a table with meta data for every treenode ID (first column). The
78-
remaining columns are <em>parent-id</em>, <em># presynaptic sites</em>,
79-
<em># postsynaptic sites</em>, <em>x</em>, <em>y</em> and <em>z</em>. The
80-
root node has no parent and its entry will have <em>null</em> in the
81-
corresponding field in the meta data file.
76+
Along those files two meta data files are created, named <em>nodes.csv</em>
77+
and <em>metadata.json</em>. The former contains a table with meta data for
78+
every treenode ID (first column). The remaining columns are
79+
<em>parent-id</em>, <em># presynaptic sites</em>, <em># postsynaptic
80+
sites</em>, <em>x</em>, <em>y</em> and <em>z</em>. The root node has no
81+
parent and its entry will have <em>null</em> in the corresponding field in
82+
the meta data file. The latter metadata file contains information on the
83+
project, image stacks and exporting user.
8284
</li>
8385
<li>
8486
<a id='export-connector-archive' href='#'>

0 commit comments

Comments
 (0)