Skip to content

Commit bd66603

Browse files
committed
First example of splitted provenance
1 parent 1202d77 commit bd66603

12 files changed

+347
-0
lines changed

examples/dcm2niix/README.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# BIDS Prov example for `dcm2niix`
2+
3+
This example aims at showing the capture of provenance for a [`dcm2niix`](https://github.com/rordenlab/dcm2niix) usecase: converting DICOM data to Nifti files within a BIDS dataset.
4+
5+
Source data for this example can be found here: https://github.com/psychoinformatics-de/hirni-demo. This is a datalad dataset containing anatomical and functional MRI acquisitions. The contents of this dataset can be downloaded using:
6+
```shell
7+
mkdir sourcedata
8+
cd sourcedata
9+
datalad install --recursive https://github.com/psychoinformatics-de/hirni-demo.git
10+
```
11+
12+
> [!NOTE] Note that the dataset must be added inside the `sourcedata/` directory.
13+
14+
15+
## Experiment #1
16+
17+
We describe the provenance records inside the following files:
18+
```
19+
.
20+
├── prov
21+
│ ├── bidsprov_context.json
22+
│ ├── environments.prov.jsonld
23+
│ └── software.prov.jsonld
24+
└── sub_02
25+
├── ses_20130717141500
26+
│ └── anat
27+
│ └── sub-02_ses-20130717141500_T1w.prov.jsonld
28+
└── ses_20140425155335
29+
└── func
30+
└── sub-02_ses-20140425155335_task-oneback_run-1_bold.prov.jsonld
31+
```
32+
33+
* `sub-02_ses-20130717141500_T1w.prov.jsonld` and `sub-02_ses-20140425155335_task-oneback_run-1_bold.prov.jsonld` are sidecars defining provenance for the corresponding `.nii` files.
34+
* `bidsprov_context.json` is the BIDS Prov context from https://purl.org/nidash/bidsprov/context.json, with minor changes.
35+
* `environments.prov.jsonld` mutualises the declaration of software environments objects for lower level prov files
36+
* `software.prov.jsonld` mutualises the declaration of software pieces objects for lower level prov files
37+
38+
The python script `code/exp1_merge_prov.py` aims at merging all these provenance records into one RFD graph.
39+
40+
```shell
41+
pip install -r code/requirements.txt
42+
mkdir prov/experiment_1/
43+
python code/exp1_merge_prov.py
44+
```
45+
46+
From that, we generate the RDF graph `prov/experiment_1/merged_provenance.ttl`.
47+
48+
Notice that starting from line 50 of the file, the generated triplets are not correct.
49+
```turtle
50+
[] prov:Activity <urn:conversion-5a66f5be> ;
51+
```
52+
53+
This is probably due to the fact that rdflib 7.1.3 does not conform with JSON-LD 1.1 and is therefore not able to parse graphs using [type indexing](https://www.w3.org/TR/json-ld11/#example-106-indexing-data-in-json-ld-by-type) as we used it e.g. in the `Records_hirni-demo_software` term of the `prov/software.prov.jsonld` file.
54+
Furthermore, all the triplets describing the type of node objects are missing because of rdflib not understanding type indexing.
55+
56+
As a workaround, replaced the incorrect lines (50 to the end) by triplets describing the type of the objects in the merged document, as presented in `prov/experiment_1/merged_provenance_2.ttl`.
57+
From that we were able to plot the following graph `prov/experiment_1/merged_provenance_2.ttl`.
58+
![](/examples/dcm2niix/prov/experiment_1/merged_provenance_2.ttl)
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
from rdflib import Graph
2+
3+
graph = Graph()
4+
graph.parse("prov/software.prov.jsonld", format = "json-ld")
5+
graph.parse("prov/environments.prov.jsonld", format = "json-ld")
6+
graph.parse("sub_02/ses_20130717141500/anat/sub-02_ses-20130717141500_T1w.prov.jsonld", format = "json-ld")
7+
graph.parse("sub_02/ses_20140425155335/func/sub-02_ses-20140425155335_task-oneback_run-1_bold.prov.jsonld", format = "json-ld")
8+
print(len(graph))
9+
graph.serialize(destination = "prov/experiment_1/merged_provenance.ttl")
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
rdflib==7.1.3
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{
2+
"@context": [
3+
"bidsprov_context.json",
4+
{
5+
"Records_hirni-demo_environments": "Records"
6+
}
7+
],
8+
"BIDSProvVersion": "0.0.1",
9+
"Records_hirni-demo_environments": {
10+
"Environments": [
11+
{
12+
"Id": "fedora",
13+
"Label": "Fedora release 36 (Thirty Six)",
14+
"OperatingSystem": " GNU/Linux 6.2.15-100.fc36.x86_64"
15+
}
16+
]
17+
}
18+
}
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
@prefix prov: <http://www.w3.org/ns/prov#> .
2+
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
3+
4+
<bids::sub-02/ses-20130717141500/anat/sub-02_ses-20130717141500_T1w.json> rdfs:label "sub-02_ses-20130717141500_T1w.json" ;
5+
prov:wasGeneratedBy <urn:conversion-00f3a18f> .
6+
7+
<bids::sub-02/ses-20130717141500/anat/sub-02_ses-20130717141500_T1w.nii> rdfs:label "sub-02_ses-20130717141500_T1w.nii" ;
8+
prov:wasGeneratedBy <urn:conversion-00f3a18f> .
9+
10+
<bids::sub-02/ses-20140425155335/func/sub-02_ses-20140425155335_task-oneback_run-1_bold.json> rdfs:label "sub-02_ses-20140425155335_task-oneback_run-1_bold.json" ;
11+
prov:wasGeneratedBy <urn:conversion-5a66f5be> .
12+
13+
<bids::sub-02/ses-20140425155335/func/sub-02_ses-20140425155335_task-oneback_run-1_bold.nii> rdfs:label "sub-02_ses-20140425155335_task-oneback_run-1_bold.nii" ;
14+
prov:wasGeneratedBy <urn:conversion-5a66f5be> .
15+
16+
<file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/fmriprep-4a586b50> rdfs:label "fmriprep" ;
17+
prov:Revision "v1.0.20220720" .
18+
19+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0001.dcm> rdfs:label "N2D_0001.dcm" .
20+
21+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0002.dcm> rdfs:label "N2D_0002.dcm" .
22+
23+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0003.dcm> rdfs:label "N2D_0003.dcm" .
24+
25+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042547586> rdfs:label "MR.1.3.46.670589.11.38317.5.0.4476.2014042516042547586" .
26+
27+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042548588> rdfs:label "MR.1.3.46.670589.11.38317.5.0.4476.2014042516042548588" .
28+
29+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042551590> rdfs:label "MR.1.3.46.670589.11.38317.5.0.4476.2014042516042551590" .
30+
31+
<file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/dcm2niix-4a586b50> rdfs:label "dcm2niix" ;
32+
prov:Revision "v1.0.20220720" .
33+
34+
<file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/fedora> rdfs:label "Fedora release 36 (Thirty Six)" .
35+
36+
<urn:conversion-00f3a18f> rdfs:label "Conversion" ;
37+
prov:used <bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0001.dcm>,
38+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0002.dcm>,
39+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0003.dcm>,
40+
<file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/fedora> ;
41+
prov:wasAssociatedWith <file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/dcm2niix-4a586b50> .
42+
43+
<urn:conversion-5a66f5be> rdfs:label "Conversion" ;
44+
prov:used <bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042547586>,
45+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042548588>,
46+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042551590>,
47+
<file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/fedora> ;
48+
prov:wasAssociatedWith <file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/dcm2niix-4a586b50> .
49+
50+
[] prov:Activity <urn:conversion-5a66f5be> ;
51+
prov:Entity <bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042547586>,
52+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042548588>,
53+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042551590>,
54+
<bids::sub-02/ses-20140425155335/func/sub-02_ses-20140425155335_task-oneback_run-1_bold.json>,
55+
<bids::sub-02/ses-20140425155335/func/sub-02_ses-20140425155335_task-oneback_run-1_bold.nii> .
56+
57+
[] prov:Activity <urn:conversion-00f3a18f> ;
58+
prov:Entity <bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0001.dcm>,
59+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0002.dcm>,
60+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0003.dcm>,
61+
<bids::sub-02/ses-20130717141500/anat/sub-02_ses-20130717141500_T1w.json>,
62+
<bids::sub-02/ses-20130717141500/anat/sub-02_ses-20130717141500_T1w.nii> .
63+
64+
[] prov:Entity <file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/fedora> .
65+
66+
[] prov:Agent <file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/dcm2niix-4a586b50>,
67+
<file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/fmriprep-4a586b50> .
68+
184 KB
Loading
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
@prefix prov: <http://www.w3.org/ns/prov#> .
2+
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
3+
4+
<bids::sub-02/ses-20130717141500/anat/sub-02_ses-20130717141500_T1w.json> rdfs:label "sub-02_ses-20130717141500_T1w.json" ;
5+
prov:wasGeneratedBy <urn:conversion-00f3a18f> .
6+
7+
<bids::sub-02/ses-20130717141500/anat/sub-02_ses-20130717141500_T1w.nii> rdfs:label "sub-02_ses-20130717141500_T1w.nii" ;
8+
prov:wasGeneratedBy <urn:conversion-00f3a18f> .
9+
10+
<bids::sub-02/ses-20140425155335/func/sub-02_ses-20140425155335_task-oneback_run-1_bold.json> rdfs:label "sub-02_ses-20140425155335_task-oneback_run-1_bold.json" ;
11+
prov:wasGeneratedBy <urn:conversion-5a66f5be> .
12+
13+
<bids::sub-02/ses-20140425155335/func/sub-02_ses-20140425155335_task-oneback_run-1_bold.nii> rdfs:label "sub-02_ses-20140425155335_task-oneback_run-1_bold.nii" ;
14+
prov:wasGeneratedBy <urn:conversion-5a66f5be> .
15+
16+
<file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/fmriprep-4a586b50> rdfs:label "fmriprep" ;
17+
prov:Revision "v1.0.20220720" .
18+
19+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0001.dcm> rdfs:label "N2D_0001.dcm" .
20+
21+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0002.dcm> rdfs:label "N2D_0002.dcm" .
22+
23+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0003.dcm> rdfs:label "N2D_0003.dcm" .
24+
25+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042547586> rdfs:label "MR.1.3.46.670589.11.38317.5.0.4476.2014042516042547586" .
26+
27+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042548588> rdfs:label "MR.1.3.46.670589.11.38317.5.0.4476.2014042516042548588" .
28+
29+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042551590> rdfs:label "MR.1.3.46.670589.11.38317.5.0.4476.2014042516042551590" .
30+
31+
<file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/dcm2niix-4a586b50> rdfs:label "dcm2niix" ;
32+
prov:Revision "v1.0.20220720" .
33+
34+
<file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/fedora> rdfs:label "Fedora release 36 (Thirty Six)" .
35+
36+
<urn:conversion-00f3a18f> rdfs:label "Conversion" ;
37+
prov:used <bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0001.dcm>,
38+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0002.dcm>,
39+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0003.dcm>,
40+
<file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/fedora> ;
41+
prov:wasAssociatedWith <file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/dcm2niix-4a586b50> .
42+
43+
<urn:conversion-5a66f5be> rdfs:label "Conversion" ;
44+
prov:used <bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042547586>,
45+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042548588>,
46+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042551590>,
47+
<file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/fedora> ;
48+
prov:wasAssociatedWith <file:///dev/BEP028_BIDSprov/examples/dcm2niix/prov/dcm2niix-4a586b50> .
49+
50+
51+
<urn:conversion-00f3a18f> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Activity .
52+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0001.dcm> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Entity .
53+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0002.dcm> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Entity .
54+
<bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0003.dcm> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Entity .
55+
<bids::sub-02/ses-20130717141500/anat/sub-02_ses-20130717141500_T1w.json> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Entity .
56+
<bids::sub-02/ses-20130717141500/anat/sub-02_ses-20130717141500_T1w.nii> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Entity .
57+
58+
<file:///dev/tests/bids_prov/split_data/prov/dcm2niix-4a586b50> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Agent .
59+
<file:///dev/tests/bids_prov/split_data/prov/fmriprep-4a586b50> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Agent .
60+
61+
<file:///dev/tests/bids_prov/split_data/prov/fedora> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Entity .
62+
63+
<urn:conversion-5a66f5be> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Activity .
64+
65+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042547586> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Entity .
66+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042548588> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Entity .
67+
<bids::sourcedata/hirni-demo/acq2/dicoms/example-dicom-functional-master/dicoms/MR.1.3.46.670589.11.38317.5.0.4476.2014042516042551590> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Entity .
68+
<bids::sub-02/ses-20140425155335/func/sub-02_ses-20140425155335_task-oneback_run-1_bold.json> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Entity .
69+
<bids::sub-02/ses-20140425155335/func/sub-02_ses-20140425155335_task-oneback_run-1_bold.nii> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> prov:Entity .
70+
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
{
2+
"@context": [
3+
"bidsprov_context.json",
4+
{
5+
"Records_hirni-demo_software": "Records"
6+
}
7+
],
8+
"BIDSProvVersion": "0.0.1",
9+
"Records_hirni-demo_software": {
10+
"Software": [
11+
{
12+
"Id": "dcm2niix-4a586b50",
13+
"Label": "dcm2niix",
14+
"Version": "v1.0.20220720"
15+
},
16+
{
17+
"Id": "fmriprep-4a586b50",
18+
"Label": "fmriprep",
19+
"Version": "v1.0.20220720"
20+
}
21+
]
22+
}
23+
}

examples/dcm2niix/sub_02/ses_20130717141500/anat/sub-02_ses-20130717141500_T1w.nii

Whitespace-only changes.
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
{
2+
"@context": [
3+
"../../../prov/bidsprov_context.json",
4+
{
5+
"@base": "../../../prov/",
6+
"Records_sub-02_ses-20130717141500_T1w": "Records"
7+
}
8+
],
9+
"BIDSProvVersion": "0.0.1",
10+
"Records_sub-02_ses-20130717141500_T1w": {
11+
"Activities": [
12+
{
13+
"Id": "urn:conversion-00f3a18f",
14+
"Label": "Conversion",
15+
"Command": "dcm2niix -o . -f sub-%i/ses-%t/anat/sub-%i_ses-%t_T1w sourcedata/acq1/dicoms/example-dicom-structural-master/dicoms/",
16+
"AssociatedWith": "dcm2niix-4a586b50",
17+
"Used": [
18+
"fedora",
19+
"bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0001.dcm",
20+
"bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0002.dcm",
21+
"bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0003.dcm"
22+
]
23+
}
24+
],
25+
"Entities": [
26+
{
27+
"Id": "bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0001.dcm",
28+
"Label": "N2D_0001.dcm"
29+
},
30+
{
31+
"Id": "bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0002.dcm",
32+
"Label": "N2D_0002.dcm"
33+
},
34+
{
35+
"Id": "bids::sourcedata/hirni-demo/acq1/dicoms/example-dicom-structural-master/dicoms/N2D_0003.dcm",
36+
"Label": "N2D_0003.dcm"
37+
},
38+
{
39+
"Id": "bids::sub-02/ses-20130717141500/anat/sub-02_ses-20130717141500_T1w.nii",
40+
"Label": "sub-02_ses-20130717141500_T1w.nii",
41+
"GeneratedBy": "urn:conversion-00f3a18f"
42+
},
43+
{
44+
"Id": "bids::sub-02/ses-20130717141500/anat/sub-02_ses-20130717141500_T1w.json",
45+
"Label": "sub-02_ses-20130717141500_T1w.json",
46+
"GeneratedBy": "urn:conversion-00f3a18f"
47+
}
48+
]
49+
}
50+
}

0 commit comments

Comments
 (0)