5 3D Models from Medical Imaging
We present a use case, which shows how to derive a 3D model from a
series of diagnostic images. The procedure here described is used to
give an overview of the main functionalities of the software and the
basic techniques of images and model management with the aim of
providing a generic workflow, and some insights useful for the optimal
resolution of specific cases.
The case presented here shows how to get a model of the mandible and how
to prepare it for printing. The presented procedure is well suited to
the extraction of models of organs with a strong contrast to surrounding
tissue, such as bone tissue in CT.
Before starting to work with images we must ensure that these do not
contain sensitive data with which we can trace the patient’s identity.
In this case we will use the open-source software DICOM Confidential,
developed by the team Data Intensive Research of the University of
Edinburgh 46 146. The software allows you to
load a folder containing the images, which are then processed according
to the defined directives.
When the software is opened, the graphical user interface (GUI) appears.
Here is possible to upload the folder containing the set to be
anonymized.
The entries Policy URI and Workflow file have to be filled. These
are the indications on the workflow to be performed and on the
anonymisation operations to be executed on the images. The software
provides standard templates to be used. These can be modified as
required, and are located in the path | C: Userpath DICOM Confidential
| (where |C: Userpath DICOM Confidential | is the folder where the
software was installed).
The output path is | C: Userpath DICOM Confidential data ANONYMISED
|. Using the standard configuration files, each anonymized set will be
recognized by the DICOM reading software (3D Slicer in our case) as
belonging to the single patient “ANONYMISED”, to which all the
anonymized studies will be added. To be taken into account during the
organization of the system.
Is possible to download anonymised datasets from The Cancer Imaging
Archive 47, a database maintained by the The National
Cancer Institute (NCI) and the University of Arkansas for Medical
Sciences (UAMS) to support multidisciplinary research, together with
the Cancer Genome Atlas dataset 48 49.
Several studies with various imaging techniques are available, which
give the opportunity to deepen the analysis of various segments of the
body and the oncological diseases that may occur. To use the images,
follow the simple procedures described on the website. Once the data is
obtained, it can be loaded directly onto the visualization software.
The images obtained can then be uploaded to 3D Slicer for viewing and
processing. We will use a set of images downloaded from The Cancer
Imaging Archive portal, located within the | TCGA - HNSC collection,
Subject ID: TGCA-BA-6868, scan: Neck BW Axial|.
The software uses the DICOM module 50 to load and manage
diagnostic image sets. The Volume Rendering module can be used to
display a rendered image.
With the Crop Volume module it is possible to cut the volume part of
our interest (Region of Interest, ROI), to lighten the computer from
data that are not needed at the moment; it will be very useful later
when we work with the models. In this example we want to create a model
of the mandibular bone, so we orientate the ROI in a way that
contains the maxillary bones, and apply the modification.
We can then start the segmentation, using the Segment Editor module.
This module contains several tools that allow us to highlight the areas
of the images of our interest. Since we want to extract the model of a
bone, the mandible, a quick method is to use the bone density range to
quickly highlight the region of our interest. Then with the Add button
we add a segment and select the Threshold tool which allows us to
select a density range to highlight. The Data Probe tool at the bottom
left gives us information about the point in the image where the mouse
tip is located, and contains an entry indicating the density at the
point. Selecting the threshold we look for a compromise between
completeness in capturing details and cleaning in the segmentation mask.
We have the possibility to perform a manual finishing of the
segmentation, so we can leave some unselected area from the threshold to
have a greater cleaning between the parts to be segmented. We consider
that the selected areas must be separated to be treated as different
segment, except if a different segment is used for each area; in that
case the selection voxels can be adjacent without merging the volumes
into a single object.
However, it is important to select the areas of interest in the best
way, and in the CT we are segmenting a delicate point for the correct
jaw separation is in the area of the condyles, where the selection is
continuous with that of the anterior wall of the glenoid cavity of the
temporal bone. Another point that have to be separated is the posterior
region of the dental arch, at the level of the occlusal plane, where the
teeth of the jaw are probably joined to the antagonists.
Once the threshold selection has been applied, we see that a mask has
been created on the voxels that fall within the selected density range.
This mask can be modified manually, adding any missing areas and
separating the parts of our interest from the adjacent ones. By clicking
on the button Show 3D we can see the preview of the 3D model created
by the segmentation; this option is useful to see if the selected areas
correspond to the model we want to obtain, but when modifying the mask
it is better to deactivate the option to save computational resources,
and activate the 3D visualization only when necessary.
The removal of the contact areas between two selected region, in our
case the mandibular condyle and temporal condyle, is performed in 3D
Slicer to speed up the subsequent processing of the model. A model could
have been created after the use of the threshold, but the separation of
two models in software such as Blender is longer and more complex than
the simple selection/deselection of voxels that can be performed in 3D
Slicer.
We therefore aim to obtain from Slicer a 3D model as precise and clean
as possible, to reduce the duration of subsequent steps, but also to
create good quality segmentation datasets, which are useful for other
purposes (data analysis, training set for Neural Network).
The Erase tool allows to remove the voxels that do not need to be
selected from the mask.
The Paint tool allows the selection of voxels of interest that are not
automatically selected by the threshold.
After finishing, we can use the Island tool with the Keep Selected
Island option enabled; clicking on the jaw selection mask. If this
island is separated from the rest of the selection we will have
correctly performed the separation, which we can control by showing the
model with the button Show 3D. If we also want the model of the rest
of the selection, we can click on the Undo button to go back one step
and retrieve the island containing the jaw and part of the skull.
We can create two separated segment: one for the mandible and one for
the skull. We add a segment from the Add button and we select it;
using the Island tool with the Add Selected Island option we click
on the skull mask and it will be added to the new segment. Separating
objects into segments causes the spatial relationship between the parts
to be lost, so if the parts need to be in a particular relation the
Merge into single file option must be selected to export the model as
a single .stl file. It is then possible to separate this single model in
its two components with the Blender software.
Then go to the module Segmentation, where from the menu on the left we
find the window Export to File; select the destination folder and
export the files in .stl format.
We import the model obtained in to MeshMixer and in the Analysis menu we select the Inspector tool. This function essentially has the task of detecting faults that must be resolved in order to have a closed model (manifold) and to remove components separated from the main mesh. It is useful for preparing a model for 3D printing, where it is necessary that the model is precisely closed, watertight. This tool is useful for solving small problems with the mesh, but complex situations may require manual repair, which can be performed in this software as well as with Blender and MeshLab.
The model can then be exported to Blender to perform a smoothing. In
Blender Modifiers menu, select the Smooth modifier. This is a
function that smooths the surface of the object, causing a slight
decrease in the object volume. In the instrument menu we can set the
number of filter passages on the model; we find a compromise between
smoothing and maintaining the original model dimensions. Another way to
smooth the surface is to do it manually, with the sculpting tools
sculpting present in both Blender and MeshMixer.
After the model smoothing we inspect it to evaluate its quality; MeshLab
can be used to compare two meshes. In our case we will measure the
difference between the original model and the model obtained after 50
iterations of the smooth modifier in Blender. When the models are
aligned with each other, the procedure is to use the Hausdorff
Distance filter to evaluate the distance between the two meshes
90, 91. The software will return measurements
related to the mesh sampling.
Hausdorff Distance computed
Sampled 4657134 pts (rng: 0) on smooth_clean_Segmentation_bone.stl
searched closest on clean_Segmentation_bone.stl
min: 0.000000; max: 0.695190; mean: 0.116956; RMS: 0.154653
Values w.r.t. BBox Diag (181.472809)
min: 0.000000; max: 0.003831; mean: 0.000644; RMS: 0.000852
Applied filter Hausdorff Distance in 29689 msec
Quality Range: 0.000000 0.666229; Used (0.006929 0.335713)
percentile (5.000000 95.000000)
Applied filter Colorize by vertex Quality in 51 msec
The mesh can then be colored to visually evaluate the discrepancy with
the original mesh. To do this from the Filter menu, select Color
Creation and Processing -> Colorize by Vertex Quality. We can
render the histogram from the menu Render -> Show Quality
Histogram.
Keep in mind that the color scale goes from red to blue, where the red
indicates maximum correspondence with the original mesh, while the
blue indicates greater distance from the original mesh. The value is
shown in model units, in this case millimeters.
As an alternative to MeshMixer it is possible to use MeshLab to perform
more advanced mesh operations. To remove regions separated from the main
mesh, use the Filter -> Cleaning and Repairing -> Remove
Isolated Pieces tool, setting the minimum number of faces at a fairly
high level, based on the number of faces shown in the bottom tool-bar.
To obtain a 2-manifold mesh 92, 93, in
practical terms a closed mesh: from the menu Filter -> Cleaning
and Repairing use the functions: Remove Duplicate Faces, Remove
Duplicate Vertex, Remove Unreferenced Vertices, Remove Faces From
Non Manifold Mesh, Remove t-vertices From Non Manifold Edges.
These commands clean up the mesh; to rebuild the 2-manifold mesh we use
the command Filter -> Remeshing, Simplification and
Reconstruction -> Screened Poisson Surface Sampling
95, 96. This clean, 2-manifold reconstruction
can be used for further processing. By performing an assessment of the
quality of the reconstruction, it is noted that the agreement with the
original mesh is very high.
Hausdorff Distance computed
Sampled 4195438 pts (rng: 0) on Poisson mesh searched
closest on Segmentation_bone.stl
min : 0.000000 max 0.432402 mean : 0.019149 RMS : 0.031849
Values w.r.t. BBox Diag (182.226028)
min : 0.000000 max 0.002373 mean : 0.000105 RMS : 0.000175
Applied filter Hausdorff Distance in 20254 msec
We can make additional checks with the MeshMixer Inspector tool to assess the need for further mesh refinements. The model must then be exported in .stl format to prepare it for 3D printing.
The digital model of the Jaw was obtained from the TC and processed to
ensure a model suitable for printing. We will be slicing the model using
the Cura software.
Load the model and select the Print Setup > Custom entry, to have
the ability to fine-tune the printing parameters, which are initially
hidden and must be activated.
The settings to be adjusted depend on several variables, including:
-
the characteristics of the printer;
-
the characteristics of the model to be printed;
-
the printing material;
-
the accuracy and the mechanical properties expected from the object that we approach to print.
The environmental conditions in which you work are also to be taken into
account, because for example some parameters may change between a print
executed in a closed printer or in an open one. The calibration of the
printer and the knowledge of its specifications are fundamental for good
printing result. Cura will generate printing instructions for the
printer based on the printer specifications we entered in the Printers
> Machine Settings menu. These settings are often updated with the
addition of the most popular printer data, but for self-assembled
printers the parameters must be entered manually.
With the calibrated printer and the information correctly entered in
Cura, the model will be displayed in the software and will be sliced
with the parameters entered. Once the slicing procedure has been
completed we will be able to see the model layer by layer (Layer
View). The so obtained G-code can be used by the printer to
produce the model.
