Skip to content

Commit 54a14b9

Browse files
bclenetbclenet
committed
Third example for dcm2niix
1 parent 8d99b3d commit 54a14b9

File tree

13 files changed

+234
-15
lines changed

13 files changed

+234
-15
lines changed

‎examples/dcm2niix_1/README.md‎

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -92,13 +92,3 @@ bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms
9292
```
9393

9494
although it is not allowed by the current version of the BIDS Prov specification to have directories as `Entities`.
95-
96-
### `TaskName` not generated
97-
98-
As specified in [this issue](https://github.com/rordenlab/dcm2niix/issues/148), `dcm2niix` is not able to propagate the value of `TaskName` (name of the task in the case of task-fMRI) automatically because this information is not in the dicom metadata.
99-
100-
In our case, the following line must be added manually in the `sub-02_ses-20140425155335_task-oneback_run-1_bold.json` file:
101-
102-
```json
103-
"TaskName": "oneback",
104-
```

‎examples/dcm2niix_1/sub-02/anat/sub-02_T1w.json‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,5 @@
1919
0.991414,
2020
-0.128044 ],
2121
"ConversionSoftware": "dcm2niix",
22-
"ConversionSoftwareVersion": "v1.0.20220720",
23-
"GeneratedBy": "bids::prov/conversion-00f3a18f"
22+
"ConversionSoftwareVersion": "v1.0.20220720"
2423
}

‎examples/dcm2niix_2/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ In this example, we rely on the fact that nodes defined in the `prov/*.prov.json
5959

6060
### Limitations
6161

62-
The `bids::prov/f#edora-b7hmkmqd` node defined in `prov/environments.prov.jsonld` is defined as an `Entity` as the current context (commit [ce0eb77](https://github.com/bids-standard/BEP028_BIDSprov/commit/ce0eb774abd9527e594bd69212a87d5047864678)) does not define the `Environments` term.
62+
The `bids::prov/#fedora-b7hmkmqd` node defined in `prov/environments.prov.jsonld` is defined as an `Entity` as the current context (commit [ce0eb77](https://github.com/bids-standard/BEP028_BIDSprov/commit/ce0eb774abd9527e594bd69212a87d5047864678)) does not define the `Environments` term.
6363

6464
Listing all the DICOM files used by the dcm2niix conversion steps would lower readability of the JSON-LD provenance files. Therefore we only listed the following directories as `Entities`:
6565
* `bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms`

‎examples/dcm2niix_2/sub-02/anat/sub-02_T1w.json‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,5 @@
1919
0.991414,
2020
-0.128044 ],
2121
"ConversionSoftware": "dcm2niix",
22-
"ConversionSoftwareVersion": "v1.0.20220720",
23-
"GeneratedBy": "bids::prov/conversion-00f3a18f"
22+
"ConversionSoftwareVersion": "v1.0.20220720"
2423
}

‎examples/dcm2niix_3/README.md‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
# BIDS Prov example for `dcm2niix`
2+
3+
This example aims at showing the capture of provenance for a [`dcm2niix`](https://github.com/rordenlab/dcm2niix) usecase: converting DICOM data to Nifti files within a BIDS dataset.
4+
5+
Source data for this example can be found here: https://github.com/psychoinformatics-de/hirni-demo. This is a datalad dataset containing anatomical and functional MRI acquisitions. The contents of this dataset can be downloaded using:
6+
7+
```shell
8+
mkdir sourcedata
9+
cd sourcedata
10+
datalad install --recursive https://github.com/psychoinformatics-de/hirni-demo.git
11+
```
12+
13+
> [!NOTE] Note that the dataset must be added inside the `sourcedata/` directory.
14+
15+
## Purpose
16+
17+
The aim of the example is to describe the provenance records using BIDS Prov, but inside several *JSON* files.
18+
We use:
19+
20+
* the `GeneratedBy` field of JSON sidecars, already existing in the BIDS specification
21+
* modality agnostic files inside the `prov/` directory
22+
23+
as follows:
24+
25+
```
26+
.
27+
├── prov
28+
│ ├── prov-dcm2niix_act.prov.json
29+
│ ├── prov-dcm2niix_base.prov.json
30+
│ ├── prov-dcm2niix_env.prov.json
31+
│ └── prov-dcm2niix_soft.prov.json
32+
└── sub-02
33+
├── ses-20130717141500
34+
│ └── anat
35+
│ └── sub-02_ses-20130717141500_T1w.json
36+
└── ses-20140425155335
37+
└── func
38+
└── sub-02_ses-20140425155335_task-oneback_run-1_bold.json
39+
```
40+
41+
## New features for BIDS / BIDS Prov
42+
43+
We introduce the following BIDS entity that is currently not existing:
44+
* `prov`
45+
* Full name: Provenance traces
46+
* Format: `prov-<label>`
47+
* Definition: A grouping of provenance traces. Defining multiple provenance traces groups is appropriate when several processings have been performed on data.
48+
49+
We introduce the following BIDS suffixes that are currently not existing:
50+
* `act`: the file describes BIDS Prov `Activities` for the group of provenance traces
51+
* `soft`: the file describes BIDS Prov `Software` for the group of provenance traces
52+
* `env`: the file describes BIDS Prov `Environments` for the group of provenance traces
53+
* `base`: the file describes common BIDS Prov parameters for the group of provenance traces (version and context for BIDS Prov)
54+
55+
We use the `GeneratedBy` field of JSON sidecars to link to `Activities` that created the file the sidecars refers to.
56+
57+
## Merging JSON in a JSON-LD file and plotting graph
58+
59+
The python script `code/merge_prov.py` aims at merging all these provenance records into one JSON-LD graph.
60+
61+
```shell
62+
mkdir prov/merged/
63+
python code/merge_prov.py
64+
```
65+
66+
From that, we generate the JSON-LD graph `prov/merge/prov-dcm2niix.prov.jsonld`. Then we were able to plot the graph as a png file. We used this command:
67+
68+
```shell
69+
pip install bids-prov==0.1.0
70+
bids_prov_visualizer --input_file prov/merged/prov-dcm2niix.prov.jsonld --output_file prov/merged/prov-dcm2niix.prov.png
71+
```
72+
73+
![](/examples/dcm2niix/prov/merged/prov-dcm2niix.prov.png)
74+
75+
### Notes
76+
77+
In this example, we rely on the fact that nodes defined in the `prov/*.prov.jsonld` files have `bids::prov/` as base IRIs.
78+
79+
The `code/merge_prov.py` code is responsible for:
80+
* merging the JSON provenance traces into the base JSON-LD graph;
81+
* create an `Entity` and linking it to the `Activity` described by the `GeneratedBy` field in the case of JSON sidecars.
82+
83+
### Limitations
84+
85+
1. The `Environments` term is not defined in the current BIDS Prov context, hence we define environments as `Entities`.
86+
87+
2. Listing all the DICOM files used by the dcm2niix conversion steps would lower readability of the JSON-LD provenance files. Therefore we only listed the following directories as `Entities`:
88+
* `bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms`
89+
* `bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms`
90+
91+
although it is not allowed by the current version of the BIDS Prov specification to have directories as `Entities`.
92+
93+
3. In this example, the provenance for JSON sidecars files is not described.

‎examples/dcm2niix_3/code/merge_prov.py‎

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
#!/usr/bin/python
2+
# coding: utf-8
3+
4+
""" Merge available prov JSON files into one RDF graph """
5+
6+
import json
7+
from pathlib import Path
8+
9+
# List of available prov files
10+
prov_soft_files = [
11+
'prov/prov-dcm2niix_soft.prov.json'
12+
]
13+
prov_env_files = [
14+
'prov/prov-dcm2niix_env.prov.json'
15+
]
16+
prov_act_files = [
17+
'prov/prov-dcm2niix_act.prov.json'
18+
]
19+
sidecar_files = [
20+
'sub-02/anat/sub-02_T1w.json'
21+
]
22+
23+
# Base jsonld
24+
base_provenance = {
25+
"Records": {
26+
"Software": [],
27+
"Activities": [],
28+
"Entities": []
29+
}
30+
}
31+
32+
# Add context and version
33+
with open('prov/prov-dcm2niix_base.prov.json', encoding = 'utf-8') as file:
34+
base_provenance.update(json.load(file))
35+
36+
# Parse Software
37+
for prov_file in prov_soft_files:
38+
with open(prov_file, encoding = 'utf-8') as file:
39+
data = json.load(file)
40+
for key, value in data.items():
41+
value['Id'] = key
42+
base_provenance['Records']['Software'].append(value)
43+
44+
# Parse Environments
45+
for prov_file in prov_env_files:
46+
with open(prov_file, encoding = 'utf-8') as file:
47+
data = json.load(file)
48+
for key, value in data.items():
49+
value['Id'] = key
50+
# /!\ Workaround: environments are added in the Entities list because
51+
# the Environments term is not defined in the BIDS Prov context yet
52+
base_provenance['Records']['Entities'].append(value)
53+
54+
# Parse Activities
55+
for prov_file in prov_act_files:
56+
with open(prov_file, encoding = 'utf-8') as file:
57+
data = json.load(file)
58+
for key, value in data.items():
59+
value['Id'] = key
60+
base_provenance['Records']['Activities'].append(value)
61+
62+
# Parse Sidecar files
63+
for sidecar_file in sidecar_files:
64+
# Identify data file(s) associated with the sidecar
65+
sidecar_filename = Path(sidecar_file)
66+
data_files = Path('').glob(f'{sidecar_filename.with_suffix("")}.*')
67+
data_files = [str(f) for f in list(data_files) if str(sidecar_filename) not in str(f)]
68+
69+
# Write provenance
70+
with open(sidecar_file, encoding = 'utf-8') as file:
71+
data = json.load(file)
72+
if 'GeneratedBy' in data:
73+
activity_id = data['GeneratedBy']
74+
for data_file in data_files:
75+
base_provenance['Records']['Entities'].append({
76+
"Id": f"bids::{data_file}",
77+
"GeneratedBy": activity_id
78+
})
79+
80+
# Write jsonld
81+
with open('prov/merged/prov-dcm2niix.prov.jsonld', 'w', encoding = 'utf-8') as file:
82+
file.write(json.dumps(base_provenance, indent = 2))

‎examples/dcm2niix_3/prov/prov-dcm2niix_act.prov.json‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
{
2+
"bids::prov/#conversion-00f3a18f": {
3+
"Label": "Conversion",
4+
"Command": "dcm2niix -o . -f sub-%i/anat/sub-%i_T1w sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms",
5+
"AssociatedWith": "bids::prov/#dcm2niix-khhkm7u1",
6+
"Used": [
7+
"bids::prov/#fedora-uldfv058",
8+
{
9+
"Id": "bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms",
10+
"Type": "Entity",
11+
"Label": "dicoms"
12+
}
13+
]
14+
}
15+
}

‎examples/dcm2niix_3/prov/prov-dcm2niix_base.prov.json‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
{
2+
"@context": "https://purl.org/nidash/bidsprov/context.json",
3+
"BIDSProvVersion": "0.0.1"
4+
}

‎examples/dcm2niix_3/prov/prov-dcm2niix_env.prov.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"bids::prov/#fedora-uldfv058": {
3+
"Label": "Fedora release 36 (Thirty Six)",
4+
"OperatingSystem": "GNU/Linux 6.2.15-100.fc36.x86_64"
5+
}
6+
}

‎examples/dcm2niix_3/prov/prov-dcm2niix_soft.prov.json‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"bids::prov/#dcm2niix-khhkm7u1": {
3+
"Label": "dcm2niix",
4+
"Version": "v1.0.20220720"
5+
}
6+
}

0 commit comments

Comments
 (0)