Add Mayo Reader #1321

adler-j · 2018-04-05T10:56:02Z

This adds a reader for Mayo clinic data.

Some examples:

It seems like the reconstruction is quite good as compared to the reference, so we should be "set".

adler-j

Minor stuff

adler-j · 2018-04-05T10:57:17Z

odl/contrib/datasets/ct/mayo.py

+
+    Returns
+    -------
+    partition : odl.:::


Fix comment

adler-j · 2018-04-05T10:57:29Z

odl/contrib/datasets/ct/mayo.py

+
+if __name__ == '__main__':
+    from odl.util.testutils import run_doctests
+    run_doctests()


Missing newline

adler-j · 2018-04-05T10:57:51Z

odl/contrib/datasets/ct/examples/mayo_reconstruct.py

+coords = [0, None, None]
+fbp_result.show('Recon (sagittal)', clim=[0.7, 1.3], coords=coords)
+ref.show('Reference (sagittal)', clim=[0.7, 1.3], coords=coords)
+(ref - fbp_result).show('Diff (sagittal)', clim=[-0.1, 0.1], coords=coords)


adler-j · 2018-04-05T10:58:23Z

odl/contrib/datasets/ct/examples/mayo_reconstruct.py

+from odl.contrib.datasets.ct.mayo import (projections_from_folder,
+                                          volume_from_folder)
+
+mayo_folder = 'E:/Data/MayoClinic'  # replace with your local folder


How should we handle stuff like this? We cannot possibly provide the data due to licence issues.

For the string: just a dot?
On the bigger point, I think this is the only sensible option.

Ok, I'll change to that

kohr-h

Not a whole lot to complain about, I'd say fix those and go ahead.

kohr-h · 2018-04-12T19:56:22Z

odl/contrib/datasets/README.md

@@ -21,6 +21,11 @@ Reference datasets with accompanying ODL geometries etc.
    CT data as provided by FIPS. The data is non-human and high resolution.
    * `walnut_data`
    * `lotus_root_data`
+
+
+    CT data as provided by Mayo Clinic. The data is human and high resolution. To access the data, see https://www.aapm.org/GrandChallenge/LowDoseCT/#registration


"The data is human" - errr what?

Latest trend in big data

Haha, yes that was my idea as well

kohr-h · 2018-04-12T20:03:38Z

odl/contrib/datasets/ct/examples/mayo_reconstruct.py

+"""Reconstruct Mayo dataset using FBP and compare to reference recon.
+
+Note that this example requires that Mayo has been previously downloaded and is
+stored in "mayo_folder".


I prefer the more correct term "directory" which can be abbreviated to the even shorter "dir". "folder" is so windows.

Also, this may be misunderstood as literally a directory called "mayo_folder". Thus, mention that it's the variable with that name which counts.

kohr-h · 2018-04-12T20:08:04Z

odl/contrib/datasets/README.md

@@ -21,6 +21,11 @@ Reference datasets with accompanying ODL geometries etc.
    CT data as provided by FIPS. The data is non-human and high resolution.
    * `walnut_data`
    * `lotus_root_data`
+
+
+    CT data as provided by Mayo Clinic. The data is human and high resolution. To access the data, see https://www.aapm.org/GrandChallenge/LowDoseCT/#registration


Maybe you can also write a word or two on what you have to do to get the data (no details, just that it requires registration because it's potentially sensitive). I often find it frustrating to just be pointed to a website, only to find out 10 minutes later that I can't get it easily.

I agree we should be explicit about that stuff

kohr-h · 2018-04-12T20:09:34Z

odl/contrib/datasets/ct/examples/mayo_reconstruct.py

+from odl.contrib.datasets.ct.mayo import (projections_from_folder,
+                                          volume_from_folder)
+
+mayo_folder = 'E:/Data/MayoClinic'  # replace with your local folder


For the string: just a dot?
On the bigger point, I think this is the only sensible option.

kohr-h · 2018-04-12T20:10:12Z

odl/contrib/datasets/ct/examples/mayo_reconstruct.py

+volume_folder = mayo_folder + '/Training Cases/L067/full_1mm_sharp'
+partition, volume = volume_from_folder(volume_folder)
+
+# Load projection data


a subset of the

kohr-h · 2018-04-12T20:21:09Z

odl/contrib/datasets/ct/mayo.py

+        data_array = data_array.reshape([rows, cols], order='F').T
+
+        # Rescale array
+        # rescaled_array = rescale_intercept + data_array * rescale_slope


hu_factor missing. But I think the code is pretty obvious, the comment is not really needed

kohr-h · 2018-04-12T20:25:32Z

odl/contrib/datasets/ct/mayo.py

+    proj_start : int
+        Index of the first projection to use. Used for subsampling.
+    proj_end : int
+        Index of the final projection to use.


This reads as inclusive, but the implementation is exclusive. Also the value -1 needs explanation.

Comment was a leftover before I added the HU scaling

kohr-h · 2018-04-12T20:26:25Z

odl/contrib/datasets/ct/mayo.py

+    geometry : ConeFlatGeometry
+        Geometry corresponding to the Mayo projector.
+    proj_data : `numpy.ndarray`
+        Projection data, normalized such that the reconstruction is g/cm^3.


will have unit ...

Generally great to mention this!

kohr-h · 2018-04-12T20:41:46Z

odl/contrib/datasets/ct/mayo.py

+    Parameters
+    ----------
+    folder : str
+        Path to the folder where the dicom files are stored.


kohr-h · 2018-04-12T20:48:47Z

odl/contrib/datasets/ct/mayo.py

+        # Get data array and convert to correct coordinates
+        data_array = np.array(np.frombuffer(dataset.PixelData, 'H'),
+                              dtype='float32')
+        data_array = data_array.reshape([rows, cols], order='F').T


This is a complicated way of writing

data_array.reshape([cols, rows], order='C')

adler-j · 2018-05-08T12:15:25Z

Will merge after CI

adler-j · 2018-05-08T12:31:47Z

Getting "The command "if [[ "$BUILD_DOCS" == "true" ]]; then cd $TRAVIS_BUILD_DIR/doc/source && python generate_doc.py && cd -; travis-sphinx build -n -s $TRAVIS_BUILD_DIR/doc/source 2>/dev/null; fi" exited with 1.", anyone has any idea as to why?

aringh · 2018-05-09T09:41:01Z

I got the same in #1340. I thought it was due to the new documentation, (using r""" and removing double backslashes if I understood correctly), but I did not have time to investigate it further. However, you do not seem to have that, so maybe it is something else.

aringh · 2018-05-09T09:45:44Z

I had a similar problem in #1267 (I think it was...) which resulted in a lot of commits where I removed the suppressing of the output from the build in Travis. I do not recall exactly what I did; after squashing it all became this commit. However, in the process I got a lot of help from @kohr-h

kohr-h · 2018-05-09T17:38:45Z

See #1344

adler-j mentioned this pull request Apr 5, 2018

ENH: Add mayo clinic reader #420

Closed

adler-j force-pushed the mayo branch from c2cf681 to 611ada4 Compare April 6, 2018 15:34

adler-j commented Apr 6, 2018

View reviewed changes

kohr-h approved these changes Apr 12, 2018

View reviewed changes

adler-j force-pushed the mayo branch from 611ada4 to 0acd805 Compare May 3, 2018 06:21

kohr-h mentioned this pull request May 3, 2018

Release 0.7.0 #1335

Closed

25 tasks

adler-j force-pushed the mayo branch from 0acd805 to b47a4fb Compare May 8, 2018 12:15

aringh mentioned this pull request May 9, 2018

Add posibility to turn off color bar in 2d plot #1343

Merged

adler-j added 5 commits May 22, 2018 15:15

Fix overflow in size, closes odlgroup#1320

cdfa5c8

ENH: Add test for size of tensor

041d5ee

ENH: Add mayo clinic reader, closes odlgroup#420

deb7dad

MAINT: Style fixes to mayo reader

bdd9770

DOC: change dicom -> DICOM

29b4f75

adler-j force-pushed the mayo branch from b47a4fb to 29b4f75 Compare May 22, 2018 14:08

TST: Remove randomly failing test in power_method_opnorm

e8136a2

adler-j merged commit 2f713f8 into odlgroup:master May 22, 2018

adler-j deleted the mayo branch May 22, 2018 16:31

aringh mentioned this pull request Jun 16, 2018

Add functionality to read Mayo clinic data #418

Closed

Add Mayo Reader #1321

Add Mayo Reader #1321

Uh oh!

Conversation

adler-j commented Apr 5, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

adler-j left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kohr-h left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adler-j commented May 8, 2018

Uh oh!

adler-j commented May 8, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aringh commented May 9, 2018

Uh oh!

aringh commented May 9, 2018

Uh oh!

kohr-h commented May 9, 2018

Uh oh!

Uh oh!

adler-j commented Apr 5, 2018 •

edited

Loading

adler-j commented May 8, 2018 •

edited

Loading