Skip to content

Home

Jump to bottom
Andy Gilbert edited this page Jan 25, 2021 · 4 revisions

Overview

This repo is generally divided into 3 separate steps:

  1. (VTK Slicing) A 2D slice is extracted from a 3D VTK model and saved as an image.
  2. (Pseudo Images) Rotation, cropping, masking, and other operations are applied to the image to convert it to the desired form.
  3. (Saving) The images are saved.

The following describes the options used to control the flow as well as each step in more detail.

Note that currently this repo has only been tested on Linux (Ubuntu 18.04)

Options

Although options were kept as simple as possible, the variety of extraction methods available inevitably requires a complex control set. All options are contained in export_config.py and are organized on several levels.

At the top level is an argparse ArgumentParser. Run python create_seg_datset.py -h to get a full overview of these options. Some options control which of the 3 steps should be run, others control extraction parameters such as the view or number of images, and others control debugging options.

There are also specific options for each view. Generally these options control the necessary transformations to be applied in step 2. These are defined within export_config.py since these should not generally change between runs. As new views are added new options will be needed here.

Finally there are several dataset specific parameters which can be modified to change the script for new dataset types. These include things such as the ratio of full chamber to LV focused images for apical 4 chamber/2 chamber views.

VTK Slicing

The first step in the pipeline is to pull slices out from the VTK based models. This action is handled by the VTKSlicer class. This class is controlled by the options class discussed above. The input is a directory (--vtk_data_dir) from which all .vtk files will be collected (up to --max_models).

The class relies on two functions which are determined based on the --view_name option. The first is a landmark finder. This class finds 3 landmark points which are used to define the plane that will be extracted. For instance for an apical four chamber image it finds the mitral and tricuspid valves and the apex. The second is an iterator function. Typically several different slices are extracted from the same model (set by --num_slices) to extend the dataset. This class defines how the original base plane can be modified to extract new planes. This consists of defining axes of rotation. The ranges of rotation are controlled by flags in the main argument parser (--x_axis_rotation_param and --y_axis_rotation_param). Note that the parameter --rotation_type can also be used to either extract iterations from a minimum to maximum angle (--rotation_type "iterate") or random rotations with a given standard deviation (--rotation_type "random")

To view how the extracted slices are oriented compared to each other, set the flags --save_vtk_slices and --disable_align_vtk_slices. This will save a .vtk file for each 2D slice which can then be opened in ParaView with the correct orientation. These options should not be set for final extraction since not aligning the slices will cause the export to .png images to fail.

A VTKSliceDataframe should be initialized to keep track of the exported slices and this dataframe is then passed into the create_slices function of the VTKSlicer. This function will automatically save the dataframe using the export options set in the argparse object. Specifically it will be saved to "(--output_dir)/(--name)/vtk_df_X") where X is the dataframe number. When extracting many slices the size of the dataframe can become large so the VTKSliceDataframe will automatically save and increment X after a certain number of rows are reached. The dataframe will save both an Excel (.xlsx) and pickle (.pck) version of each dataframe. The excel version contains the parameters for easy viewing while the pickle version contains the actual image data.

The actual slicing logic is contained in the VTKSlicer._create_slice function.

Pseudo Image Extraction

This step will extract one or more pseudo images for each vtk slice created above (number controlled by --images_per_slice option). The script will automatically iterate through all the vtk dataframes created above and create a corresponding pseudo dataframe containing the images. As shown in the get_pseudo_imgs_from_vtk function in create_seg_dataset.py and in a similar fashion to the first step above, a PseudoDatasetDataFrame should first be created to store all of the built pseudo images.

The main class to handle the pseudo transformation is SliceHandler. In contrast to the process above, a new slice handler should be initialized for each row in the vtk dataframe. This is because most of the transforms are applied in the __init__ function of the SliceHandler class. The transforms consist of several steps:

  • First, a set of alignment transformas are applied. These apply a generic rotation and shift to generally align slices in teh correct direction
  • Second, blood pools labels are added (_add_blodd_pools()). These are computed using the rules about the surrounding tissue types.
  • Third, modifiers are applied. These are functions which may modify the slice e.g. by increasing the thickness of the pericardium for PLAX views
  • Fourth, a set of custom transforms are applied. These are applied to shift the image to match the typical distribution of real images.

The most complex part of these is the fourth step, the custom transforms. These transforms are defined in image_transforms.py and are defined in the style of PyTorch transforms. In other words each is a class with an __init__ method that defines necessary parameters and a __call__ method which is called for each image and applies the actual transform.

Note that these transforms currently apply the transform directly to the image. This can lead to bugs if one transform crops the image and then another later tries to expand it. There are some workarounds in the code now, but in the future it should be modified so that the transforms instead just modify a bounding box and then the bounding box is applied at the end.

After these transformations are applied the create_pseudo_images function can be called. This generates a pseudo image by adding/multiplying uniform noise and blurring the image. A set of appearance transforms for each view is also defined in export_config.py (pseudo_transforms). Then a cone is generated and used to mask the image.

Note that a DEBUG_WITH_IMAGES flag is at the top of slice_editor.py which can be used to plot images at various stages. For more fine-grained debugging a DEBUG_WITH_IMAGES flag is also in image_transforms.py which will plot an image after every transform is applied.

Image Saving

This step simply saves the pseudo and label images created in the previous step. One function (process_output_image) is called to change the label values to a wide range from 0-255 for easy visualization.

Final label values are:

label_background=40
lv_blood_pool=70
rv_blood_pool=100
la_blood_pool=130
ra_blood_pool=160
lv_myocardium=190
la_myocardium=200
other_tissue=220
pericardium=250
Clone this wiki locally