The fovea is an important clinical landmark that is used as a reference for assessing various quantitative measures, such as central retinal thickness or drusen count. In this paper we propose a novel method for automatic detection of the foveal center in Optical Coherence Tomography (OCT) scans. Although the clinician will generally aim to center the OCT scan on the fovea, post-acquisition image processing will give a more accurate estimate of the true location of the foveal center. A Convolutional Neural Network (CNN) was trained on a set of 781 OCT scans that classifies each pixel in the OCT B-scan with a probability of belonging to the fovea. Dilated convolutions were used to obtain a large receptive field, while maintaining pixel-level accuracy. In order to train the network more effectively, negative patches were sampled selectively after each epoch. After CNN classification of the entire OCT volume, the predicted foveal center was chosen as the voxel with maximum output probability, after applying an optimized three-dimensional Gaussian blurring. We evaluate the performance of our method on a data set of 99 OCT scans presenting different stages of Age-related Macular Degeneration (AMD). The fovea was correctly detected in 96:9% of the cases, with a mean distance error of 73 μm(±112 μm). This result was comparable to the performance of a second human observer who obtained a mean distance error of 69 μm (±94 μm). Experiments showed that the proposed method is accurate and robust even in retinas heavily affected by pathology.

Automatically detecting anatomy orientation is an important task in medical image analysis. Specifically, the ability to automatically detect coarse orientation of structures is useful to minimize the effort of fine/accurate orientation detection algorithms, to initialize non-rigid deformable registration algorithms or to align models to target structures in model-based segmentation algorithms. In this work, we present a deep convolution neural network (DCNN)-based method for fast and robust detection of the coarse structure orientation, i.e., the hemi-sphere where the principal axis of a structure lies. That is, our algorithm predicts whether the principal orientation of a structure is in the northern hemisphere or southern hemisphere, which we will refer to as UP and DOWN, respectively, in the remainder of this manuscript. The only assumption of our method is that the entire structure is located within the scan’s field-of-view (FOV). To efficiently solve the problem in 3D space, we formulated it as a multi-planar 2D deep learning problem. In the training stage, a large number coronal-sagittal slice pairs are constructed as 2-channel images to train a DCNN to classify whether a scan is UP or DOWN. During testing, we randomly sample a small number of coronal-sagittal 2-channel images and pass them through our trained network. Finally, coarse structure orientation is determined using majority voting. We tested our method on 114 Elbow MR Scans. Experimental results suggest that only five 2-channel images are sufficient to achieve a high success rate of 97.39%. Our method is also extremely fast and takes approximately 50 milliseconds per 3D MR scan. Our method is insensitive to the location of the structure in the FOV.

Representation learning through deep learning (DL) architecture has shown tremendous potential for identification, local- ization, and texture classification in various medical imaging modalities. However, DL applications to segmentation of objects especially to deformable objects are rather limited and mostly restricted to pixel classification. In this work, we propose marginal shape deep learning (MaShDL), a framework that extends the application of DL to deformable shape segmentation by using deep classifiers to estimate the shape parameters. MaShDL combines the strength of statistical shape models with the automated feature learning architecture of DL. Unlike the iterative shape parameters estimation approach of classical shape models that often leads to a local minima, the proposed framework is robust to local minima optimization and illumination changes. Furthermore, since the direct application of DL framework to a multi-parameter estimation problem results in a very high complexity, our framework provides an excellent run-time performance solution by independently learning shape parameter classifiers in marginal eigenspaces in the decreasing order of variation. We evaluated MaShDL for segmenting the lung field from 314 normal and abnormal pediatric chest radiographs and obtained a mean Dice similarity coefficient of 0:927 using only the four highest modes of variation (compared to 0:888 with classical ASM1 (p-value=0:01) using same configuration). To the best of our knowledge this is the first demonstration of using DL framework for parametrized shape learning for the delineation of deformable objects.

Accurate segmentation of lung fields on chest radiographs is the primary step for computer-aided detection of various conditions such as lung cancer and tuberculosis. The size, shape and texture of lung fields are key parameters for chest X-ray (CXR) based lung disease diagnosis in which the lung field segmentation is a significant primary step. Although many methods have been proposed for this problem, lung field segmentation remains as a challenge. In recent years, deep learning has shown state of the art performance in many visual tasks such as object detection, image classification and semantic image segmentation. In this study, we propose a deep convolutional neural network (CNN) framework for segmentation of lung fields. The algorithm was developed and tested on 167 clinical posterior-anterior (PA) CXR images collected retrospectively from picture archiving and communication system (PACS) of Geisinger Health System. The proposed multi-scale network is composed of five convolutional and two fully connected layers. The framework achieved IOU (intersection over union) of 0.96 on the testing dataset as compared to manual segmentation. The suggested framework outperforms state of the art registration-based segmentation by a significant margin. To our knowledge, this is the first deep learning based study of lung field segmentation on CXR images developed on a heterogeneous clinical dataset. The results suggest that convolutional neural networks could be employed reliably for lung field segmentation.

The vertebral column is a complex anatomical construct, composed of vertebrae and intervertebral discs (IVDs) supported by ligaments and muscles. During life, all components undergo degenerative changes, which may in some cases cause severe, chronic and debilitating low back pain. The main diagnostic challenge is to locate the pain generator, and degenerated IVDs have been identified to act as such. Accurate and robust segmentation of IVDs is therefore a prerequisite for computer-aided diagnosis and quantification of IVD degeneration, and can be also used for computer-assisted planning and simulation in spinal surgery. In this paper, we present a novel fully automated framework for supervised segmentation of IVDs from three-dimensional (3D) magnetic resonance (MR) spine images. By considering global intensity appearance and local shape information, a landmark-based approach is first used for the detection of IVDs in the observed image, which then initializes the segmentation of IVDs by coupling deformable models with convolutional networks (ConvNets). For this purpose, a 3D ConvNet architecture was designed that learns rich high-level appearance representations from a training repository of IVDs, and then generates spatial IVD probability maps that guide deformable models towards IVD boundaries. By applying the proposed framework to 15 3D MR spine images containing 105 IVDs, quantitative comparison of the obtained against reference IVD segmentations yielded an overall mean Dice coefficient of 92.8%, mean symmetric surface distance of 0.4 mm and Hausdorff surface distance of 3.7 mm.

The role of medical imaging in global health systems is literally fundamental. Like labs, medical images are used at one point or another in almost every high cost, high value episode of care. CT scans, mammograms, and x-rays, for example, “atlas” the body and help chart a course forward for a patient’s care team. Imaging precision has improved as a result of technological advancements and breakthroughs in related medical research. Those advancements also bring with them exponential growth in medical imaging data. As IBM trains Watson to "see" medical images, Ms. Le Grand will discuss recent advances made by Watson Health and explore the potential value of "augmented intelligence" to assist healthcare providers like radiologists and cardiologists, as well as the patients they serve.

There are growing applications of quantitative computed tomography for assessment of pulmonary diseases by characterizing lung parenchyma as well as the bronchial tree. Many large multi-center studies incorporating lung imaging as a study component are interested in phenotypes relating airway branching patterns, wall-thickness, and other morphological measures. To our knowledge, there are no fully automated airway tree segmentation methods, free of the need for user review. Even when there are failures in a small fraction of segmentation results, the airway tree masks must be manually reviewed for all results which is laborious considering that several thousands of image data sets are evaluated in large studies. In this paper, we present a CT-based novel airway tree segmentation algorithm using iterative multi-scale leakage detection, freezing, and active seed detection. The method is fully automated requiring no manual inputs or post-segmentation editing. It uses simple intensity based connectivity and a new leakage detection algorithm to iteratively grow an airway tree starting from an initial seed inside the trachea. It begins with a conservative threshold and then, iteratively shifts toward generous values. The method was applied on chest CT scans of ten non-smoking subjects at total lung capacity and ten at functional residual capacity. Airway segmentation results were compared to an expert’s manually edited segmentations. Branch level accuracy of the new segmentation method was examined along five standardized segmental airway paths (RB1, RB4, RB10, LB1, LB10) and two generations beyond these branches. The method successfully detected all branches up to two generations beyond these segmental bronchi with no visual leakages.

Automatic spleen segmentation on CT is challenging due to the complexity of abdominal structures. Multi-atlas segmentation (MAS) has shown to be a promising approach to conduct spleen segmentation. To deal with the substantial registration errors between the heterogeneous abdominal CT images, the context learning method for performance level estimation (CLSIMPLE) method was previously proposed. The context learning method generates a probability map for a target image using a Gaussian mixture model (GMM) as the prior in a Bayesian framework. However, the CLSSIMPLE typically trains a single GMM from the entire heterogeneous training atlas set. Therefore, the estimated spatial prior maps might not represent specific target images accurately. Rather than using all training atlases, we propose an adaptive GMM based context learning technique (AGMMCL) to train the GMM adaptively using subsets of the training data with the subsets tailored for different target images. Training sets are selected adaptively based on the similarity between atlases and the target images using cranio-caudal length, which is derived manually from the target image. To validate the proposed method, a heterogeneous dataset with a large variation of spleen sizes (100 cc to 9000 cc) is used. We designate a metric of size to differentiate each group of spleens, with 0 to 100 cc as small, 200 to 500cc as medium, 500 to 1000 cc as large, 1000 to 2000 cc as XL, and 2000 and above as XXL. From the results, AGMMCL leads to more accurate spleen segmentations by training GMMs adaptively for different target images.

Non-invasive spleen volume estimation is essential in detecting splenomegaly. Magnetic resonance imaging (MRI) has been used to facilitate splenomegaly diagnosis in vivo. However, achieving accurate spleen volume estimation from MR images is challenging given the great inter-subject variance of human abdomens and wide variety of clinical images/modalities. Multi-atlas segmentation has been shown to be a promising approach to handle heterogeneous data and difficult anatomical scenarios. In this paper, we propose to use multi-atlas segmentation frameworks for MRI spleen segmentation for splenomegaly. To the best of our knowledge, this is the first work that integrates multi-atlas segmentation for splenomegaly as seen on MRI. To address the particular concerns of spleen MRI, automated and novel semi-automated atlas selection approaches are introduced. The automated approach interactively selects a subset of atlases using selective and iterative method for performance level estimation (SIMPLE) approach. To further control the outliers, semi-automated craniocaudal length based SIMPLE atlas selection (L-SIMPLE) is proposed to introduce a spatial prior in a fashion to guide the iterative atlas selection. A dataset from a clinical trial containing 55 MRI volumes (28 T1 weighted and 27 T2 weighted) was used to evaluate different methods. Both automated and semi-automated methods achieved median DSC > 0.9. The outliers were alleviated by the L-SIMPLE (≈1 min manual efforts per scan), which achieved 0.9713 Pearson correlation compared with the manual segmentation. The results demonstrated that the multi-atlas segmentation is able to achieve accurate spleen segmentation from the multi-contrast splenomegaly MRI scans.

Shape from shading (SFS) has been studied for decades; nevertheless, its overly simple assumptions and its ill-conditioning have resulted in infrequent use in real applications. Price et al. recently developed an iterative scheme named shape from motion and shading (SFMS) that models both shape and reflectance of an unknown surface simultaneously. SFMS produces a fairly accurate, dense 3D reconstruction from each frame of a pharyngeal endoscopic video, albeit with inconsistency between the 3D reconstructions of different frames. We present a comprehensive study of the SFMS scheme and several improvements to it: (1) We integrate a deformable registration method into the iterative scheme and use the fusion of multiple surfaces as a reference surface to guide the next iteration’s reconstruction. This can be interpreted as incorporating regularity of a frame’s reconstruction with that of temporally nearby frames. (2) We show that the reflectance model estimation is crucial and very sensitive to noise in the data. Moreover, even when the surface reflection is not assumed to be Lambertian, the reflectance model estimation function in SFMS is still overly simple for endoscopy of human tissue. By removing outlier pixels, by preventing unrealistic BRDF estimation, and by reducing the falloff speed of illumination in SFS to account for the effect of multiple bouncing of the light, we improve the reconstruction accuracy.

Optical coherence tomography (OCT) yields high-resolution, three-dimensional images of the retina. A better understanding of retinal nerve fiber bundle (RNFB) trajectories in combination with visual field data may be used for future diagnosis and monitoring of glaucoma. However, manual tracing of these bundles is a tedious task. In this work, we present an automatic technique to estimate the orientation of RNFBs from volumetric OCT scans. Our method consists of several steps, starting from automatic segmentation of the RNFL. Then, a stack of en face images around the posterior nerve fiber layer interface was extracted. The image showing the best visibility of RNFB trajectories was selected for further processing. After denoising the selected en face image, a semblance structure-oriented filter was applied to probe the strength of local linear structure in a discrete set of orientations creating an orientation space. Gaussian filtering along the orientation axis in this space is used to find the dominant orientation. Next, a confidence map was created to supplement the estimated orientation. This confidence map was used as pixel weight in normalized convolution to regularize the semblance filter response after which a new orientation estimate can be obtained. Finally, after several iterations an orientation field corresponding to the strongest local orientation was obtained. The RNFB orientations of six macular scans from three subjects were estimated. For all scans, visual inspection shows a good agreement between the estimated orientation fields and the RNFB trajectories in the en face images. Additionally, a good correlation between the orientation fields of two scans of the same subject was observed. Our method was also applied to a larger field of view around the macula. Manual tracing of the RNFB trajectories shows a good agreement with the automatically obtained streamlines obtained by fiber tracking.

Fluorescence microscopy is used to image multiple subcellular structures in living cells which are not readily observed using conventional optical microscopy. Moreover, two-photon microscopy is widely used to image structures deeper in tissue. Recent advancement in fluorescence microscopy has enabled the generation of large data sets of images at different depths, times, and spectral channels. Thus, automatic object segmentation is necessary since manual segmentation would be inefficient and biased. However, automatic segmentation is still a challenging problem as regions of interest may not have well defined boundaries as well as non-uniform pixel intensities. This paper describes a method for segmenting tubular structures in fluorescence microscopy images of rat kidney and liver samples using adaptive histogram equalization, foreground/background segmentation, steerable filters to capture directional tendencies, and connected-component analysis. The results from several data sets demonstrate that our method can segment tubular boundaries successfully. Moreover, our method has better performance when compared to other popular image segmentation methods when using ground truth data obtained via manual segmentation.

Multiple sclerosis (MS) is a multi-factorial autoimmune disorder, characterized by spatial and temporal dissemination of brain lesions that are visible in T2-weighted and Proton Density (PD) MRI. Assessment of lesion burden and is useful for monitoring the course of the disease, and assessing correlates of clinical outcomes. Although there are established semi-automated methods to measure lesion volume, most of them require human interaction and editing, which are time consuming and limits the ability to analyze large sets of data with high accuracy. The primary objective of this work is to improve existing segmentation algorithms and accelerate the time consuming operation of identifying and validating MS lesions. In this paper, a Deep Neural Network for MS Lesion Segmentation is implemented. The MS lesion samples are extracted from the Partners Comprehensive Longitudinal Investigation of Multiple Sclerosis (CLIMB) study. A set of 900 subjects with T2, PD and a manually corrected label map images were used to train a Deep Neural Network and identify MS lesions. Initial tests using this network achieved a 90% accuracy rate. A secondary goal was to enable this data repository for big data analysis by using this algorithm to segment the remaining cases available in the CLIMB repository.

Manual segmentation of brain lesions from Magnetic Resonance Images (MRI) is cumbersome and introduces errors due to inter-rater variability. This paper introduces a semi-supervised technique for detection of brain lesion from MRI using Generative Adversarial Networks (GANs). GANs comprises of a Generator network and a Discriminator network which are trained simultaneously with the objective of one bettering the other. The networks were trained using non lesion patches (n=13,000) from 4 different MR sequences. The network was trained on BraTS dataset and patches were extracted from regions excluding tumor region. The Generator network generates data by modeling the underlying probability distribution of the training data, (PData). The Discriminator learns the posterior probability P (Label Data) by classifying training data and generated data as “Real” or “Fake” respectively. The Generator upon learning the joint distribution, produces images/patches such that the performance of the Discriminator on them are random, i.e. P (Label Data = GeneratedData) = 0.5. During testing, the Discriminator assigns posterior probability values close to 0.5 for patches from non lesion regions, while patches centered on lesion arise from a different distribution (PLesion) and hence are assigned lower posterior probability value by the Discriminator. On the test set (n=14), the proposed technique achieves whole tumor dice score of 0.69, sensitivity of 91% and specificity of 59%. Additionally the generator network was capable of generating non lesion patches from various MR sequences.

Automated and reliable segmentation of subcortical structures from human brain magnetic resonance images is of great importance for volumetric and shape analyses in quantitative neuroimaging studies. However, poor boundary contrast and variable shape of these structures make the automated segmentation a tough task. We propose a 3D graph-based machine learning method, called LOGISMOS-RF, to segment the caudate and the putamen from brain MRI scans in a robust and accurate way. An atlas-based tissue classification and bias-field correction method is applied to the images to generate an initial segmentation for each structure. Then a 3D graph framework is utilized to construct a geometric graph for each initial segmentation. A locally trained random forest classifier is used to assign a cost to each graph node. The max-flow algorithm is applied to solve the segmentation problem. Evaluation was performed on a dataset of T1-weighted MRI’s of 62 subjects, with 42 images used for training and 20 images for testing. For comparison, FreeSurfer, FSL and BRAINSCut approaches were also evaluated using the same dataset. Dice overlap coefficients and surface-to-surfaces distances between the automated segmentation and expert manual segmentations indicate the results of our method are statistically significantly more accurate than the three other methods, for both the caudate (Dice: 0.89 ± 0.03) and the putamen (0.89 ± 0.03).

Sulci are groove-like regions lying in the depth of the cerebral cortex between gyri, which together, form a folded appearance in human and mammalian brains. Sulci play an important role in the structural analysis of the brain, morphometry (i.e., the measurement of brain structures), anatomical labeling and landmark-based registration.1 Moreover, sulcal morphological changes are related to cortical thickness, whose measurement may provide useful information for studying variety of psychiatric disorders. Manually extracting sulci requires complying with complex protocols, which make the procedure both tedious and error prone.2 In this paper, we describe an automatic procedure, employing geometric active contours, which extract the sulci. Sulcal boundaries are obtained by minimizing a certain energy functional whose minimum is attained at the boundary of the given sulci.

The basal ganglia and limbic system, particularly the thalamus, putamen, internal and external globus pallidus, substantia nigra, and sub-thalamic nucleus, comprise a clinically relevant signal network for Parkinson’s disease. In order to manually trace these structures, a combination of high-resolution and specialized sequences at 7T are used, but it is not feasible to scan clinical patients in those scanners. Targeted imaging sequences at 3T such as F-GATIR, and other optimized inversion recovery sequences, have been presented which enhance contrast in a select group of these structures. In this work, we show that a series of atlases generated at 7T can be used to accurately segment these structures at 3T using a combination of standard and optimized imaging sequences, though no one approach provided the best result across all structures. In the thalamus and putamen, a median Dice coefficient over 0.88 and a mean surface distance less than 1.0mm was achieved using a combination of T1 and an optimized inversion recovery imaging sequences. In the internal and external globus pallidus a Dice over 0.75 and a mean surface distance less than 1.2mm was achieved using a combination of T1 and FGATIR imaging sequences. In the substantia nigra and sub-thalamic nucleus a Dice coefficient of over 0.6 and a mean surface distance of less than 1.0mm was achieved using the optimized inversion recovery imaging sequence. On average, using T1 and optimized inversion recovery together produced significantly improved segmentation results than any individual modality (p<0.05 wilcox sign-rank test).

Brain Extraction (classification of brain and non-brain tissue) of MRI brain images is a crucial pre-processing step necessary for imaging-based anatomical studies of the human brain. Several automated methods and software tools are available for performing this task, but differences in MR image parameters (pulse sequence, resolution) and instrumentand subject-dependent noise and artefacts affect the performance of these automated methods. We describe and evaluate a method that automatically adapts the default parameters of the Brain Surface Extraction (BSE) algorithm to optimize a cost function chosen to reflect accurate brain extraction. BSE uses a combination of anisotropic filtering, Marr-Hildreth edge detection, and binary morphology for brain extraction. Our algorithm automatically adapts four parameters associated with these steps to maximize the brain surface area to volume ratio. We evaluate the method on a total of 109 brain volumes with ground truth brain masks generated by an expert user. A quantitative evaluation of the performance of the proposed algorithm showed an improvement in the mean (s.d.) Dice coefficient from 0.8969 (0.0376) for default parameters to 0.9509 (0.0504) for the optimized case. These results indicate that automatic parameter optimization can result in significant improvements in definition of the brain mask.

Crossing fibers are prevalent in human brains and a subject of intense interest for neuroscience. Diffusion tensor imaging (DTI) can resolve tissue orientation but is blind to crossing fibers. Many advanced diffusion-weighted magnetic resolution imaging (MRI) approaches have been presented to extract crossing-fibers from high angular resolution diffusion imaging (HARDI), but the relative sensitivity and specificity of approaches remains unclear. Here, we examine two leading approaches (PAS and q-ball) in the context of a large-scale, single subject reproducibility study. A single healthy individual was scanned 11 times with 96 diffusion weighted directions and 10 reference volumes for each of five b-values (1000, 1500, 2000, 2500, 3000 s/mm2) for a total of 5830 volumes (over the course of three sessions). We examined the reproducibility of the number of fibers per voxel, volume fraction, and crossing-fiber angles. For each method, we determined the minimum resolvable angle for each acquisition. Reproducibility of fiber counts per voxel was generally high (~80% consensus for PAS and ~70% for q-ball), but there was substantial bias between individual repetitions and model estimated with all data (~10% lower consensus for PAS and ~15% lower for q-ball). Both PAS and q-ball predominantly discovered fibers crossing at near 90 degrees, but reproducibility was higher for PAS across most measures. Within voxels with low anisotropy, q-ball finds more intra-voxel structure; meanwhile, PAS resolves multiple fibers at greater than 75 degrees for more voxels. These results can inform researchers when deciding between HARDI approaches or interpreting findings across studies.

We investigate the applicability of large-scale Granger Causality (lsGC) for extracting a measure of multivariate information flow between pairs of regional brain activities from resting-state functional MRI (fMRI) and test the effectiveness of these measures for predicting a disease state. Such pairwise multivariate measures of interaction provide high-dimensional representations of connectivity profiles for each subject and are used in a machine learning task to distinguish between healthy controls and individuals presenting with symptoms of HIV Associated Neurocognitive Disorder (HAND). Cognitive impairment in several domains can occur as a result of HIV infection of the central nervous system. The current paradigm for assessing such impairment is through neuropsychological testing. With fMRI data analysis, we aim at non-invasively capturing differences in brain connectivity patterns between healthy subjects and subjects presenting with symptoms of HAND. To classify the extracted interaction patterns among brain regions, we use a prototype-based learning algorithm called Generalized Matrix Learning Vector Quantization (GMLVQ). Our approach to characterize connectivity using lsGC followed by GMLVQ for subsequent classification yields good prediction results with an accuracy of 87% and an area under the ROC curve (AUC) of up to 0.90. We obtain a statistically significant improvement (p<0.01) over a conventional Granger causality approach (accuracy = 0.76, AUC = 0.74). High accuracy and AUC values using our multivariate method to connectivity analysis suggests that our approach is able to better capture changes in interaction patterns between different brain regions when compared to conventional Granger causality analysis known from the literature.

High-angular-resolution diffusion-weighted imaging (HARDI) MRI acquisitions have become common for use with higher order models of diffusion. Despite successes in resolving complex fiber configurations and probing microstructural properties of brain tissue, there is no common consensus on the optimal b-value and number of diffusion directions to use for these HARDI methods. While this question has been addressed by analysis of the diffusion-weighted signal directly, it is unclear how this translates to the information and metrics derived from the HARDI models themselves. Using a high angular resolution data set acquired at a range of b-values, and repeated 11 times on a single subject, we study how the b-value and number of diffusion directions impacts the reproducibility and precision of metrics derived from Q-ball imaging, a popular HARDI technique. We find that Q-ball metrics associated with tissue microstructure and white matter fiber orientation are sensitive to both the number of diffusion directions and the spherical harmonic representation of the Q-ball, and often are biased when under sampled. These results can advise researchers on appropriate acquisition and processing schemes, particularly when it comes to optimizing the number of diffusion directions needed for metrics derived from Q-ball imaging.

The new hybrid-BCI-DNI atlas is a high-resolution MPRAGE, single-subject atlas, constructed using both anatomical and functional information to guide the parcellation of the cerebral cortex. Anatomical labeling was performed manually on coronal single-slice images guided by sulcal and gyral landmarks to generate the original (non-hybrid) BCI-DNI atlas. Functional sub-parcellations of the gyral ROIs were then generated from 40 minimally preprocessed resting fMRI datasets from the HCP database. Gyral ROIs were transferred from the BCI-DNI atlas to the 40 subjects using the HCP grayordinate space as a reference. For each subject, each gyral ROI was subdivided using the fMRI data by applying spectral clustering to a similarity matrix computed from the fMRI time-series correlations between each vertex pair. The sub-parcellations were then transferred back to the original cortical mesh to create the subparcellated hBCI-DNI atlas with a total of 67 cortical regions per hemisphere. To assess the stability of the gyral subdivisons, a separate set of 60 HCP datasets were processed as follows: 1) coregistration of the structural scans to the hBCI-DNI atlas; 2) coregistration of the anatomical BCI-DNI atlas without functional subdivisions, followed by sub-parcellation of each subject’s resting fMRI data as described above. We then computed consistency between the anatomically-driven delineation of each gyral subdivision and that obtained per subject using individual fMRI data. The gyral sub-parcellations generated by atlas-based registration show variable but generally good overlap of the confidence intervals with the resting fMRI-based subdivisions. These consistency measures will provide a quantitative measure of reliability of each subdivision to users of the atlas.

Purpose: To develop, test, evaluate and apply a novel tool for the white matter fiber-based analysis of T1w/T2w ratio maps quantifying myelin content. Background: The cerebral white matter in the human brain develops from a mostly non-myelinated state to a nearly fully mature white matter myelination within the first few years of life. High resolution T1w/T2w ratio maps are believed to be effective in quantitatively estimating myelin content on a voxel-wise basis. We propose the use of a fiber-tract-based analysis of such T1w/T2w ratio data, as it allows us to separate fiber bundles that a common regional analysis imprecisely groups together, and to associate effects to specific tracts rather than large, broad regions. Methods: We developed an intuitive, open source tool to facilitate such fiber-based studies of T1w/T2w ratio maps. Via its Graphical User Interface (GUI) the tool is accessible to non-technical users. The framework uses calibrated T1w/T2w ratio maps and a prior fiber atlas as an input to generate profiles of T1w/T2w values. The resulting fiber profiles are used in a statistical analysis that performs along-tract functional statistical analysis. We applied this approach to a preliminary study of early brain development in neonates. Results: We developed an open-source tool for the fiber based analysis of T1w/T2w ratio maps and tested it in a study of brain development.

CT and MR perfusion weighted imaging (PWI) enable quantification of perfusion parameters in stroke studies. These parameters are calculated from the residual impulse response function (IRF) based on a physiological model for tissue perfusion. The standard approach for estimating the IRF is deconvolution using oscillatory-limited singular value decomposition (oSVD) or Frequency Domain Deconvolution (FDD). FDD is widely recognized as the fastest approach currently available for deconvolution of CT Perfusion/MR PWI. In this work, three faster methods are proposed. The first is a direct (model based) crude approximation to the final perfusion quantities (Blood flow, Blood volume, Mean Transit Time and Delay) using the Welch-Satterthwaite approximation for gamma fitted concentration time curves (CTC). The second method is a fast accurate deconvolution method, we call Analytical Fourier Filtering (AFF). The third is another fast accurate deconvolution technique using Showalter’s method, we call Analytical Showalter’s Spectral Filtering (ASSF). Through systematic evaluation on phantom and clinical data, the proposed methods are shown to be computationally more than twice as fast as FDD. The two deconvolution based methods, AFF and ASSF, are also shown to be quantitatively accurate compared to FDD and oSVD.

Coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular events (CVEs). CAC can be quantified in chest CT scans acquired in lung screening. However, in these images the reproducibility of CAC quantification is compromised by cardiac motion that occurs during scanning, thereby limiting the reproducibility of CVE risk assessment. We present a system for the identification of CACs strongly affected by cardiac motion artifacts by using a convolutional neural network (CNN). This study included 125 chest CT scans from the National Lung Screening Trial (NLST). Images were acquired with CT scanners from four different vendors (GE, Siemens, Philips, Toshiba) with varying tube voltage, image resolution settings, and without ECG synchronization. To define the reference standard, an observer manually identified CAC lesions and labeled each according to the presence of cardiac motion: strongly affected (positive), mildly affected/not affected (negative). A CNN was designed to automatically label the identified CAC lesions according to the presence of cardiac motion by analyzing a patch from the axial CT slice around each lesion. From 125 CT scans, 9201 CAC lesions were analyzed. 8001 lesions were used for training (19% positive) and the remaining 1200 (50% positive) were used for testing. The proposed CNN achieved a classification accuracy of 85% (86% sensitivity, 84% specificity). The obtained results demonstrate that the proposed algorithm can identify CAC lesions that are strongly affected by cardiac motion. This could facilitate further investigation into the relation of CAC scoring reproducibility and the presence of cardiac motion artifacts.

Echocardiography (echo) is the most common test for diagnosis and management of patients with cardiac condi- tions. While most medical imaging modalities benefit from a relatively automated procedure, this is not the case for echo and the quality of the final echo view depends on the competency and experience of the sonographer. It is not uncommon that the sonographer does not have adequate experience to adjust the transducer and acquire a high quality echo, which may further affect the clinical diagnosis. In this work, we aim to aid the operator during image acquisition by automatically assessing the quality of the echo and generating the Automatic Echo Score (AES). This quality assessment method is based on a deep convolutional neural network, trained in an end-to-end fashion on a large dataset of apical four-chamber (A4C) echo images. For this project, an expert car- diologist went through 2,904 A4C images obtained from independent studies and assessed their condition based on a 6-scale grading system. The scores assigned by the expert ranged from 0 to 5. The distribution of scores among the 6 levels were almost uniform. The network was then trained on 80% of the data (2,345 samples). The average absolute error of the trained model in calculating the AES was 0.8 ± 0:72. The computation time of the GPU implementation of the neural network was estimated at 5 ms per frame, which is sufficient for real-time deployment.

Cochlear implants (CIs) are neural prosthetics for treating severe-to-profound hearing loss. Our group has developed an image-guided cochlear implant programming (IGCIP) system that uses image analysis techniques to recommend patientspecific CI processor settings to improve hearing outcomes. One crucial step in IGCIP is the localization of CI electrodes in post-implantation CTs. Manual localization of electrodes requires time and expertise. To automate this process, our group has proposed automatic techniques that have been validated on CTs acquired with scanners that produce images with an extended range of intensity values. However, there are many clinical CTs acquired with a limited intensity range. This limitation complicates the electrode localization process. In this work, we present a pre-processing step for CTs with a limited intensity range and extend the methods we proposed for full intensity range CTs to localize CI electrodes in CTs with limited intensity range. We evaluate our method on CTs of 20 subjects implanted with CI arrays produced by different manufacturers. Our method achieves a mean localization error of 0.21mm. This indicates our method is robust for automatic localization of CI electrodes in different types of CTs, which represents a crucial step for translating IGCIP from research laboratory to clinical use.

Cystic Fibrosis (CF) results in severe bronchiectasis in nearly all cases. Bronchiectasis is a disease where parts of the airways are permanently dilated. The development and the progression of bronchiectasis is not evenly distributed over the entire lungs – rather, individual functional units are affected differently. We developed a fully automated method for the precise calculation of lobe-based airway taper indices. To calculate taper indices, some preparatory algorithms are needed. The airway tree is segmented, skeletonized and transformed to a rooted acyclic graph. This graph is used to label the airways. Then a modified version of the previously validated integral based method (IBM) for airway geometry determination is utilized. The rooted graph, the airway lumen and wall information are then used to calculate the airway taper indices. Using a computer-generated phantom simulating 10 cross sections of airways we present results showing a high accuracy of the modified IBM. The new taper index calculation method was applied to 144 volumetric inspiratory low-dose MDCT scans. The scans were acquired from 36 children with mild CF at 4 time-points (baseline, 3 month, 1 year, 2 years). We found a moderate correlation with the visual lobar Brody bronchiectasis scores by three raters (r2 = 0.36, p < .0001). The taper index has the potential to be a precise imaging biomarker but further improvements are needed. In combination with other imaging biomarkers, taper index calculation can be an important tool for monitoring the progression and the individual treatment of patients with bronchiectasis.

Analysis of retinal fundus images is essential for physicians, optometrists and ophthalmologists in the diagnosis, care and treatment of patients. The first step of almost all forms of automated fundus analysis begins with the segmentation and subtraction of the retinal vasculature, while analysis of that same structure can aid in the diagnosis of certain retinal and cardiovascular conditions, such as diabetes or stroke. This paper investigates the use of a Convolutional Neural Network as a multi-channel classifier of retinal vessels using DRIVE, a database of fundus images. The result of the network with the application of a confidence threshold was slightly below the 2nd observer and gold standard, with an accuracy of 0.9419 and ROC of 0.9707. The output of the network with on post-processing boasted the highest sensitivity found in the literature with a score of 0.9568 and a good ROC score of 0.9689. The high sensitivity of the system makes it suitable for longitudinal morphology assessments, disease detection and other similar tasks.

In this paper we propose a new strategy for the recovery of complex anatomical deformations that exhibit local discontinuities, such as the shearing found at the lung-ribcage interface, using multi-grid octree B-splines. B- spline based image registration is widely used in the recovery of respiration induced deformations between CT images. However, the continuity imposed upon the computed deformation field by the parametrizing cubic B- spline basis function results in an inability to correctly capture discontinuities such as the sliding motion at organ boundaries. The proposed technique efficiently captures deformation within and at organ boundaries without the need for prior knowledge, such as segmentation, by selectively increasing deformation freedom within image regions exhibiting poor local registration. Experimental results show that the proposed method achieves more physically plausible deformations than traditional global B-spline methods.

In this paper, we propose a 3D registration method for retinal optical coherence tomography (OCT) volumes. The proposed method consists of five main steps: First, a projection image of the 3D OCT scan is created. Second, the vessel enhancement filter is applied on the projection image to detect vessel shadow. Third, landmark points are extracted based on both vessel positions and layer information. Fourth, the coherent point drift method is used to align retinal OCT volumes. Finally, a nonrigid B-spline-based registration method is applied to find the optimal transform to match the data. We applied this registration method on 15 3D OCT scans of patients with Choroidal Neovascularization (CNV). The Dice coefficients (DSC) between layers are greatly improved after applying the nonrigid registration.

We present the Active Registration Model (ARM) that couples medical image registration with regularization using a statistical model of intensity. Inspired by Active Appearance Models (AAMs), the statistical model is embedded in the registration procedure as a regularization term that penalize differences between a target image and a synthesized model reconstruction of that image. We demonstrate that the method generalizes AAMs to 3D images, many different transformation models, and many different gradient descent optimization methods. The method is validated on magnetic resonance images of human brains.

Cochlear implant surgery is a hearing restoration procedure for patients with profound hearing loss. In this surgery, an electrode is inserted into the cochlea to stimulate the auditory nerve and restore the patient’s hearing. Clinical computed tomography (CT) images are used for planning and evaluation of electrode placement, but their low resolution limits the visualization of internal cochlear structures. Therefore, high resolution micro-CT images are used to develop atlas-based segmentation methods to extract these nonvisible anatomical features in clinical CT images. Accurate registration of the high and low resolution CT images is a prerequisite for reliable atlas-based segmentation. In this study, we evaluate and compare different non-rigid B-spline registration parameters using micro-CT and clinical CT images of five cadaveric human cochleae. The varying registration parameters are cost function (normalized correlation (NC), mutual information and mean square error), interpolation method (linear, windowed-sinc and B-spline) and sampling percentage (1%, 10% and 100%). We compare the registration results visually and quantitatively using the Dice similarity coefficient (DSC), Hausdorff distance (HD) and absolute percentage error in cochlear volume. Using MI or MSE cost functions and linear or windowed-sinc interpolation resulted in visually undesirable deformation of internal cochlear structures. Quantitatively, the transforms using 100% sampling percentage yielded the highest DSC and smallest HD (0.828±0.021 and 0.25±0.09mm respectively). Therefore, B-spline registration with cost function: NC, interpolation: B-spline and sampling percentage: moments 100% can be the foundation of developing an optimized atlas-based segmentation algorithm of intracochlear structures in clinical CT images.

Efficient Cochlear Implant (CI) surgery requires prior knowledge of the cochlea’s size and its characteristics. This information helps to select suitable implants for different patients. To get these measurements, a segmentation method of cochlea medical images is needed. An important pre-processing step for good cochlea segmentation involves efficient image registration. The cochlea’s small size and complex structure, in addition to the different resolutions and head positions during imaging, reveals a big challenge for the automated registration of the different image modalities. In this paper, an Automatic Cochlea Image Registration (ACIR) method for multi- modal human cochlea images is proposed. This method is based on using small areas that have clear structures from both input images instead of registering the complete image. It uses the Adaptive Stochastic Gradient Descent Optimizer (ASGD) and Mattes’s Mutual Information metric (MMI) to estimate 3D rigid transform parameters. The use of state of the art medical image registration optimizers published over the last two years are studied and compared quantitatively using the standard Dice Similarity Coefficient (DSC). ACIR requires only 4.86 seconds on average to align cochlea images automatically and to put all the modalities in the same spatial locations without human interference. The source code is based on the tool elastix and is provided for free as a 3D Slicer plugin. Another contribution of this work is a proposed public cochlea standard dataset which can be downloaded for free from a public XNAT server.

Whole-body diffusion-weighted (WB-DW) MRI in combination with anatomical MRI has shown a great poten- tial in bone and soft tissue tumour detection, evaluation of lymph nodes and treatment response assessment. Because of the vast body coverage, whole-body MRI is acquired in separate stations, which are subsequently combined into a whole-body image. However, inter-station and inter-modality image misalignments can occur due to image distortions and patient motion during acquisition, which may lead to inaccurate representations of patient anatomy and hinder visual assessment. Automated and accurate whole-body image formation and alignment of the multi-modal MRI images is therefore crucial. We investigated several registration approaches for the formation or stitching of the whole-body image stations, followed by a deformable alignment of the multi- modal whole-body images. We compared a pairwise approach, where diffusion-weighted (DW) image stations were sequentially aligned to a reference station (pelvis), to a groupwise approach, where all stations were simultaneously mapped to a common reference space while minimizing the overall transformation. For each, a choice of input images and corresponding metrics was investigated. Performance was evaluated by assessing the quality of the obtained whole-body images, and by verifying the accuracy of the alignment with whole-body anatomical sequences. The groupwise registration approach provided the best compromise between the formation of WB- DW images and multi-modal alignment. The fully automated method was found to be robust, making its use in the clinic feasible.

Taking a multi-objective optimization approach to deformable image registration has recently gained attention, because such an approach removes the requirement of manually tuning the weights of all the involved objectives. Especially for problems that require large complex deformations, this is a non-trivial task. From the resulting Pareto set of solutions one can then much more insightfully select a registration outcome that is most suitable for the problem at hand. To serve as an internal optimization engine, currently used multi-objective algorithms are competent, but rather inefficient. In this paper we largely improve upon this by introducing a multi-objective real-valued adaptation of the recently introduced Gene-pool Optimal Mixing Evolutionary Algorithm (GOMEA) for discrete optimization. In this work, GOMEA is tailored specifically to the problem of deformable image registration to obtain substantially improved efficiency. This improvement is achieved by exploiting a key strength of GOMEA: iteratively improving small parts of solutions, allowing to faster exploit the impact of such updates on the objectives at hand through partial evaluations. We performed experiments on three registration problems. In particular, an artificial problem containing a disappearing structure, a pair of pre- and post-operative breast CT scans, and a pair of breast MRI scans acquired in prone and supine position were considered. Results show that compared to the previously used evolutionary algorithm, GOMEA obtains a speed-up of up to a factor of ~1600 on the tested registration problems while achieving registration outcomes of similar quality.

Late Gadolinium-Enhanced Cardiac MRI (LGE CMRI) is a non-invasive technique, which has shown promise in detecting native and post-ablation atrial scarring. To visualize the scarring, a precise segmentation of the left atrium (LA) and pulmonary veins (PVs) anatomy is performed as a first step—usually from an ECG gated CMRI roadmap acquisition—and the enhanced scar regions from the LGE CMRI images are superimposed. The anatomy of the LA and PVs in particular is highly variable and manual segmentation is labor intensive and highly subjective. In this paper, we developed a multi-atlas propagation based whole heart segmentation (WHS) to delineate the LA and PVs from ECG gated CMRI roadmap scans. While this captures the anatomy of the atrium well, the PVs anatomy is less easily visualized. The process is therefore augmented by semi-automated manual strokes for PVs identification in the registered LGE CMRI data. This allows us to extract more accurate anatomy than the fully automated WHS. Both qualitative visualization and quantitative assessment with respect to manual segmented ground truth showed that our method is efficient and effective with an overall mean Dice score of 0.91.

Whole breast segmentation is the first step in quantitative analysis of breast MR images. This task is challenging due mainly to the chest-wall line’s (CWL) spatially varying appearance and nearby distracting structures, both being complex. In this paper, we propose an automatic three-dimensional (3-D) segmentation method of whole breast in sagittal MR images. This method distinguishes itself from others in two main aspects. First, it reformulates the challenging problem of CWL localization into an equivalence that searches for an optimal smooth depth field and so fully utilizes the 3-D continuity of the CWLs. Second, it employs a localized self- adapting algorithm to adjust to the CWL’s spatial variation. Experimental results on real patient data with expert-outlined ground truth show that the proposed method can segment breasts accurately and reliably, and that its segmentation is superior to that of previously established methods.

Magnetic Resonance Imaging (MRI) and Magnetic Resonance Spectroscopy (MRS) together have become the gold standard in the precise quantification of body fat. The study of the quantification of fat in the human body has matured in recent years from a simplistic interest in the whole-body fat content to detailing regional fat distributions. The realization that body-fat, or adipose tissue (AT) is far from being a mere aggregate mass or deposit but a biologically active organ in and of itself, may play a role in the association between obesity and the various pathologies that are the biggest health issues of our time. Furthermore, a major bottleneck in most medical image assessments of adipose tissue content and distribution is the lack of automated image analysis. This motivated us to develop a proper and at least partially automated methodology to accurately and reproducibly determine both body fat content and distribution in the human body, which is to be applied to cross-sectional and longitudinal studies. The AT considered here is located beneath the skin (subcutaneous) as well as around the internal organs and between muscles (visceral and inter-muscular). There are also special fat depots on and around the heart (pericardial) as well as around the aorta (peri-aortic). Our methods focus on measuring and classifying these various AT deposits in the human body in an intervention study that involves the acquisition of thoracic and abdominal MR images via a Dixon technique.

In this work, we present a formal definition and an efficient algorithm to compute the cores of Oriented Relative Fuzzy Connectedness (ORFC), a recent seed-based segmentation technique. The core is a region where the seed can be moved without altering the segmentation, an important aspect for robust techniques and reduction of user effort. We show how ORFC cores can be used to build a powerful hybrid image segmentation approach. We also provide some new theoretical relations between ORFC and Oriented Image Foresting Transform (OIFT), as well as their cores. Experimental results among several methods show that the hybrid approach conserves high accuracy, avoids the shrinking problem and provides robustness to seed placement inside the desired object due to the cores properties.

The segmentation of the thyroid in ultrasound images is a field of active research. The thyroid is a gland of the endocrine system and regulates several body functions. Measuring the volume of the thyroid is regular practice of diagnosing pathological changes. In this work, we compare three approaches for semi-automatic thyroid segmentation in freehand-tracked three-dimensional ultrasound images. The approaches are based on level set, graph cut and feature classification. For validation, sixteen 3D ultrasound records were created with ground truth segmentations, which we make publicly available. The properties analyzed are the Dice coefficient when compared against the ground truth reference and the effort of required interaction. Our results show that in terms of Dice coefficient, all algorithms perform similarly. For interaction, however, each algorithm has advantages over the other. The graph cut-based approach gives the practitioner direct influence on the final segmentation. Level set and feature classifier require less interaction, but offer less control over the result. All three compared methods show promising results for future work and provide several possible extensions.

Computerized assessments for diagnosis or malignancy grading of cyto-histopathological specimens have drawn increased attention in the field of digital pathology. Automatic segmentation of cell nuclei is a fundamental step in such automated systems. Despite considerable research, nuclei segmentation is still a challenging task due noise, nonuniform illumination, and most importantly, in 2D projection images, overlapping and touching nuclei. In most published approaches, nuclei refinement is a post-processing step after segmentation, which usually refers to the task of detaching the aggregated nuclei or merging the over-segmented nuclei. In this work, we present a novel segmentation technique which effectively addresses the problem of individually segmenting touching or overlapping cell nuclei during the segmentation process. The proposed framework is a region-based segmentation method, which consists of three major modules: i) the image is passed through a color deconvolution step to extract the desired stains; ii) then the generalized fast radial symmetry transform is applied to the image followed by non-maxima suppression to specify the initial seed points for nuclei, and their corresponding GFRS ellipses which are interpreted as the initial nuclei borders for segmentation; iii) finally, these nuclei border initial curves are evolved through the use of a statistical level-set approach along with topology preserving criteria for segmentation and separation of nuclei at the same time. The proposed method is evaluated using Hematoxylin and Eosin, and fluorescent stained images, performing qualitative and quantitative analysis, showing that the method outperforms thresholding and watershed segmentation approaches.

The identification of spondylolysis and spondylolisthesis is important in spinal diagnosis, rehabilitation, and surgery planning. Accurate and automatic detection of spinal portion with spondylolisthesis problem will significantly reduce the manual work of physician and provide a more robust evaluation for the spine condition. Most existing automatic identification methods adopted the indirect approach which used vertebrae locations to measure the spondylolisthesis. However, these methods relied heavily on automatic vertebra detection which often suffered from the pool spatial accuracy and the lack of validated pathological training samples. In this study, we present a novel spondylolisthesis detection method which can directly locate the irregular spine portion and output the corresponding grading. The detection is done by a set of learning-based detectors which are discriminatively trained by synthesized spondylolisthesis image samples. To provide sufficient pathological training samples, we used a parameterized spine model to synthesize different types of spondylolysis images from real MR/CT scans. The parameterized model can automatically locate the vertebrae in spine images and estimate their pose orientations, and can inversely alter the vertebrae locations and poses by changing the corresponding parameters. Various training samples can then be generated from only a few spine MR/CT images. The preliminary results suggest great potential for the fast and efficient spondylolisthesis identification and measurement in both MR and CT spine images.

Modeling subject-specific shape change is one of the most important challenges in longitudinal shape analysis of disease progression. Whereas anatomical change over time can be a function of normal aging, anatomy can also be impacted by disease related degeneration. Anatomical shape change may also be affected by structural changes from neighboring shapes, which may cause non-linear variations in pose. In this paper, we propose a framework to analyze disease related shape changes by coupling extrinsic modeling of the ambient anatomical space via spatiotemporal deformations with intrinsic shape properties from medial surface analysis. We compare intrinsic shape properties of a subject-specific shape trajectory to a normative 4D shape atlas representing normal aging to isolate shape changes related to disease. The spatiotemporal shape modeling establishes inter/intra subject anatomical correspondence, which in turn enables comparisons between subjects and the 4D shape atlas, and also quantitative analysis of disease related shape change. The medial surface analysis captures intrinsic shape properties related to local patterns of deformation. The proposed framework jointly models extrinsic longitudinal shape changes in the ambient anatomical space, as well as intrinsic shape properties to give localized measurements of degeneration. Six high risk subjects and six controls are randomly sampled from a Huntington’s disease image database for qualitative and quantitative comparison.

With the increasing amount of patient information that is being collected today, the idea of using this information to inform future patient care has gained momentum. In many cases, this information comes in the form of medical images. Several algorithms have been presented to automatically segment these images, and to extract structures relevant to different diagnostic or surgical procedures. Consequently, this allows us to obtain large data-sets of shapes, in the form of triangular meshes, segmented from these images. Given correspondences between these shapes, statistical shape models (SSMs) can be built using methods like Principal Component Analysis (PCA). Often, the initial correspondences between the shapes need to be improved, and SSMs can be used to improve these correspondences. However, just as often, initial segmentations also need to be improved. Unlike many correspondence improvement algorithms, which do not affect segmentation, many segmentation improvement algorithms negatively affect correspondences between shapes. We present a method that iteratively improves both segmentation as well as correspondence by using SSMs not only to improve correspondence, but also to constrain the movement of vertices during segmentation improvement. We show that our method is able to maintain correspondence while achieving as good or better segmentations than those produced by methods that improve segmentation without maintaining correspondence. We are additionally able to achieve segmentations with better triangle quality than segmentations produced without correspondence improvement.

The optic nerve (ON) is a vital structure in the human visual system and transports all visual information from the retina to the cortex for higher order processing. Due to the lack of redundancy in the visual pathway, measures of ON damage have been shown to correlate well with visual deficits. These measures are typically taken at an arbitrary anatomically defined point along the nerve and do not characterize changes along the length of the ON. We propose a fully automated, three-dimensionally consistent technique building upon a previous independent slice-wise technique to estimate the radius of the ON and surrounding cerebrospinal fluid (CSF) on high-resolution heavily T2-weighted isotropic MRI. We show that by constraining results to be three-dimensionally consistent this technique produces more anatomically viable results. We compare this technique with the previously published slice-wise technique using a short-term reproducibility data set, 10 subjects, follow-up <1 month, and show that the new method is more reproducible in the center of the ON. The center of the ON contains the most accurate imaging because it lacks confounders such as motion and frontal lobe interference. Long-term reproducibility, 5 subjects, follow-up of approximately 11 months, is also investigated with this new technique and shown to be similar to short-term reproducibility, indicating that the ON does not change substantially within 11 months. The increased accuracy of this new technique provides increased power when searching for anatomical changes in ON size amongst patient populations.

Freehand 3D ultrasound (US) consists in acquiring a US volume by moving a tracked conventional 2D probe over an area of interest. To maintain good acoustic coupling between the probe and the skin, the operator applies pressure on the skin with the probe. This pressure deforms the underlying tissues in a variable way across the excursion of the probe, which, in turn, leads to inconsistencies in the volume. To address this problem, this paper proposes a method to estimate the deformation field sustained by each image with respect to a reference deformation free image. The method is based on a 2D biomechanical model that takes into account the mechanical parameters of the tissues depicted in the image to predict a realistic deformation field. These parameters are estimated along with the deformation field such as to maximize the mutual information between the reference and the corrected images. The image is then corrected by applying the inverse deformation field. Preliminary validation was conducted with synthetic US images generated using a 3D biomechanical model. Results show that the proposed method improves image correction compared to a purely image-based method.

Segmentation is a key task in medical image analysis because its accuracy significantly affects successive steps. Automatic segmentation methods often produce inadequate segmentations, which require the user to manually edit the produced segmentation slice by slice. Because editing is time-consuming, an editing tool that enables the user to produce accurate segmentations by only drawing a sparse set of contours would be needed. This paper describes such a framework as applied to a single object. Constrained by the additional information enabled by the manually segmented contours, the proposed framework utilizes object shape statistics to transform the failed automatic segmentation to a more accurate version. Instead of modeling the object shape, the proposed framework utilizes shape change statistics that were generated to capture the object deformation from the failed automatic segmentation to its corresponding correct segmentation. An optimization procedure was used to minimize an energy function that consists of two terms, an external contour match term and an internal shape change regularity term. The high accuracy of the proposed segmentation editing approach was confirmed by testing it on a simulated data set based on 10 in-vivo infant magnetic resonance brain data sets using four similarity metrics. Segmentation results indicated that our method can provide efficient and adequately accurate segmentations (Dice segmentation accuracy increase of 10%), with very sparse contours (only 10%), which is promising in greatly decreasing the work expected from the user.

Eye diseases and visual impairment affect millions of Americans and induce billions of dollars in annual economic burdens. Expounding upon existing knowledge of eye diseases could lead to improved treatment and disease prevention. This research investigated the relationship between structural metrics of the eye orbit and visual function measurements in a cohort of 470 patients from a retrospective study of ophthalmology records for patients (with thyroid eye disease, orbital inflammation, optic nerve edema, glaucoma, intrinsic optic nerve disease), clinical imaging, and visual function assessments. Orbital magnetic resonance imaging (MRI) and computed tomography (CT) images were retrieved and labeled in 3D using multi-atlas label fusion. Based on the 3D structures, both traditional radiology measures (e.g., Barrett index, volumetric crowding index, optic nerve length) and novel volumetric metrics were computed. Using stepwise regression, the associations between structural metrics and visual field scores (visual acuity, functional acuity, visual field, functional field, and functional vision) were assessed. Across all models, the explained variance was reasonable (R2 ~ 0.1-0.2) but highly significant (p < 0.001). Instead of analyzing a specific pathology, this study aimed to analyze data across a variety of pathologies. This approach yielded a general model for the connection between orbital structural imaging biomarkers and visual function.

Tumor tracking and progression analysis using medical images is a crucial task for physicians to provide accurate and efficient treatment plans, and monitor treatment response. Tumor progression is tracked by manual measurement of tumor growth performed by radiologists. Several methods have been proposed to automate these measurements with segmentation, but many current algorithms are confounded by attached organs and vessels. To address this problem, we present a new generalized tumor propagation model considering time-series prior images and local anatomical features using a Hierarchical Hidden Markov model (HMM) for tumor tracking. First, we apply the multi-atlas segmentation technique to identify organs/sub-organs using pre-labeled atlases. Second, we apply a semi-automatic direct 3D segmentation method to label the initial boundary between the lesion and neighboring structures. Third, we detect vessels in the ROI surrounding the lesion. Finally, we apply the propagation model with the labeled organs and vessels to accurately segment and measure the target lesion. The algorithm has been designed in a general way to be applicable to various body parts and modalities. In this paper, we evaluate the proposed algorithm on lung and lung nodule segmentation and tracking. We report the algorithm’s performance by comparing the longest diameter and nodule volumes using the FDA lung Phantom data and a clinical dataset.

Representation of human tongue motion using three-dimensional vector fields over time can be used to better understand tongue function during speech, swallowing, and other lingual behaviors. To characterize the inter-subject variability of the tongue’s shape and motion of a population carrying out one of these functions it is desirable to build a statistical model of the four-dimensional (4D) tongue. In this paper, we propose a method to construct a spatio-temporal atlas of tongue motion using magnetic resonance (MR) images acquired from fourteen healthy human subjects. First, cine MR images revealing the anatomical features of the tongue are used to construct a 4D intensity image atlas. Second, tagged MR images acquired to capture internal motion are used to compute a dense motion field at each time frame using a phase-based motion tracking method. Third, motion fields from each subject are pulled back to the cine atlas space using the deformation fields computed during the cine atlas construction. Finally, a spatio-temporal motion field atlas is created to show a sequence of mean motion fields and their inter-subject variation. The quality of the atlas was evaluated by deforming cine images in the atlas space. Comparison between deformed and original cine images showed high correspondence. The proposed method provides a quantitative representation to observe the commonality and variability of the tongue motion field for the first time, and shows potential in evaluation of common properties such as strains and other tensors based on motion fields.

Accurate CT synthesis, sometimes called electron density estimation, from MRI is crucial for successful MRI-based radiotherapy planning and dose computation. Existing CT synthesis methods are able to synthesize normal tissues but are unable to accurately synthesize abnormal tissues (i.e., tumor), thus providing a suboptimal solution. We propose a multiatlas- based hybrid synthesis approach that combines multi-atlas registration and patch-based synthesis to accurately synthesize both normal and abnormal tissues. Multi-parametric atlas MR images are registered to the target MR images by multi-channel deformable registration, from which the atlas CT images are deformed and fused by locally-weighted averaging using a structural similarity measure (SSIM). Synthetic MR images are also computed from the registered atlas MRIs by using the same weights used for the CT synthesis; these are compared to the target patient MRIs allowing for the assessment of the CT synthesis fidelity. Poor synthesis regions are automatically detected based on the fidelity measure and refined by a patch-based synthesis. The proposed approach was tested on brain cancer patient data, and showed a noticeable improvement for the tumor region.

Purpose: To evaluate an efficient iterative deconvolution method (RSEMD) for improving the quantitative accuracy of previously reconstructed breast images by commercial positron emission mammography (PEM) scanner. Materials and Methods: The RSEMD method was tested on breast phantom data and clinical PEM imaging data. Data acquisition was performed on a commercial Naviscan Flex Solo II PEM camera. This method was applied to patient breast images previously reconstructed with Naviscan software (MLEM) to determine improvements in resolution, signal to noise ratio (SNR) and contrast to noise ratio (CNR.) Results: In all of the patients’ breast studies the post-processed images proved to have higher resolution and lower noise as compared with images reconstructed by conventional methods. In general, the values of SNR reached a plateau at around 6 iterations with an improvement factor of about 2 for post-processed Flex Solo II PEM images. Improvements in image resolution after the application of RSEMD have also been demonstrated. Conclusions: A rapidly converging, iterative deconvolution algorithm with a novel resolution subsets-based approach RSEMD that operates on patient DICOM images has been used for quantitative improvement in breast imaging. The RSEMD method can be applied to clinical PEM images to improve image quality to diagnostically acceptable levels and will be crucial in order to facilitate diagnosis of tumor progression at the earliest stages. The RSEMD method can be considered as an extended Richardson-Lucy algorithm with multiple resolution levels (resolution subsets).

In X-ray imaging, scatter can produce noise, artifacts, and decreased contrast. In practice, hardware such as anti-scatter grids are often used to reduce scatter. However, the remaining scatter can still be significant and additional software-based correction are desirable. Furthermore, good software solutions can potentially reduce the amount of needed anti-scatter hardware, thereby reducing cost. In this work, we developed a software correction algorithm by adapting a class of non-local image restoration techniques to scatter reduction. In this algorithm, scatter correction is formulated as a Bayesian MAP (maximum a posteriori) problem with a non-local prior, which leads to better textural detail preservation in scatter reduction. The efficacy of our algorithm is demonstrated through experimental and simulation results.

X-ray machines are widely used for medical imaging and their cost is highly dependent on their image resolution. Due to economic reasons, lower-resolution (lower-res) machines still have a lot of customers, especially in developing economies. Software based resolution enhancement can potentially enhance the capabilities of the lower-res machines without significantly increasing their cost hence, is highly desirable. In this work, we developed an algorithm for X-ray image resolution enhancement. In this algorithm, the fractal idea and cross-resolution patch matching are used to identify low-res patches that can be used as samples for high-res patch/pixel estimation. These samples are then used to generate a prior distribution and used in a Bayesian MAP (maximum a posteriori) optimization to produce the high-res image estimate. The efficacy of our algorithm is demonstrated by experimental results.

In minimally invasive surgery, image quality is a critical pre-requisite to ensure a surgeons ability to perform a procedure. In endoscopic procedures, image quality can deteriorate for a number of reasons such as fogging due to the temperature gradient after intra-corporeal insertion, lack of focus and due to smoke generated when using electro-cautery to dissect tissues without bleeding. In this paper we investigate the use of vision processing techniques to remove surgical smoke and improve the clarity of the image. We model the image formation process by introducing a haze medium to account for the degradation of visibility. For simplicity and computational efficiency we use an adapted dark-channel prior method combined with histogram equalization to remove smoke artifacts to recover the radiance image and enhance the contrast and brightness of the final result. Our initial results on images from robotic assisted procedures are promising and show that the proposed approach may be used to enhance image quality during surgery without additional suction devices. In addition, the processing pipeline may be used as an important part of a robust surgical vision pipeline that can continue working in the presence of smoke.

Multimodal groupwise registration has been of growing interest to the image processing community due to developments in scanner technologies (e.g. multiparametric MRI, DCE-CT or PET-MR) that increased both the number of modalities and number of images under consideration. In this work a novel methodology is presented for multimodal groupwise registration that is based on Laplacian eigenmaps, a nonlinear dimensionality reduction technique. Compared to recently proposed dissimilarity metrics based on principal component analysis, the proposed metric should enable a better capture of the intensity relationships between different images in the group. The metric is constructed to be the second smallest eigenvalue from the eigenvector problem defined in Laplacian eigenmaps. The method was validated in three distinct experiments: a non-linear synthetic registration experiment, the registration of quantitative MRI data of the carotid artery, and the registration of multimodal data of the brain (RIRE). The results show increased accuracy and robustness compared to other state-of-the-art groupwise registration methodologies.

4D flow MRI images make it possible to measure pulsatile blood flow inside deforming vessel, which is critical in accurate blood flow visualization, simulation, and evaluation. Such data has great potential to overcome problems in existing work, which usually does not reflect the dynamic nature of elastic vessels and blood flows in cardiac cycles. However, the 4D flow MRI data is often low-resolution and with strong noise. Due to these challenges, few efforts have been successfully conducted to extract dynamic blood flow fields and deforming artery over cardiac cycles, especially for small artery like carotid. In this paper, a robust flow feature, particularly the mean flow intensity is used to segment blood flow regions inside vessels from 4D flow MRI images in whole cardiac cycle. To estimate this flow feature more accurately, adaptive weights are added to the raw velocity vectors based on the noise strength of MRI imaging. Then, based on this feature, target arteries are tracked in at different time steps in a cardiac cycle. This method is applied to the clinical 4D flow MRI data in neck area. Dynamic vessel walls and blood flows are effectively generated in a cardiac cycle in the relatively small carotid arteries. Good image segmentation results on 2D slices are presented, together with the visualization of 3D arteries and blood flows. Evaluation of the method was performed by clinical doctors and by checking flow volume rates in the vertebral and carotid arteries.

Single image super-resolution (SR) method can generate a high-resolution (HR) image from a low-resolution (LR) image by enhancing image resolution. In medical imaging, HR images are expected to have a potential to provide a more accurate diagnosis with the practical application of HR displays. In recent years, the super-resolution convolutional neural network (SRCNN), which is one of the state-of-the-art deep learning based SR methods, has proposed in computer vision. In this study, we applied and evaluated the SRCNN scheme to improve the image quality of magnified images in chest radiographs. For evaluation, a total of 247 chest X-rays were sampled from the JSRT database. The 247 chest X-rays were divided into 93 training cases with non-nodules and 152 test cases with lung nodules. The SRCNN was trained using the training dataset. With the trained SRCNN, the HR image was reconstructed from the LR one. We compared the image quality of the SRCNN and conventional image interpolation methods, nearest neighbor, bilinear and bicubic interpolations. For quantitative evaluation, we measured two image quality metrics, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). In the SRCNN scheme, PSNR and SSIM were significantly higher than those of three interpolation methods (p<0.001). Visual assessment confirmed that the SRCNN produced much sharper edge than conventional interpolation methods without any obvious artifacts. These preliminary results indicate that the SRCNN scheme significantly outperforms conventional interpolation algorithms for enhancing image resolution and that the use of the SRCNN can yield substantial improvement of the image quality of magnified images in chest radiographs.

In this paper, a novel approach combining the active appearance model (AAM) and graph search is proposed to segment retinal layers for optic nerve head(ONH) centered optical coherence tomography(OCT) images. The method includes two parts: preprocessing and layer segmentation. During the preprocessing phase, images is first filtered for denoising, then the B-scans are flattened. During layer segmentation, the AAM is first used to obtain the coarse segmentation results. Then a multi-resolution GS–AAM algorithm is applied to further refine the results, in which AAM is efficiently integrated into the graph search segmentation process. The proposed method was tested on a dataset which contained113-D SD-OCT images, and compared to the manual tracings of two observers on all the volumetric scans. The overall mean border positioning error for layer segmentation was found to be 7.09 ± 6.18μm for normal subjects. It was comparable to the results of traditional graph search method (8.03±10.47μm) and mean inter-observer variability (6.35±6.93μm).The preliminary results demonstrated the feasibility and efficiency of the proposed method.

In many settings, multiple Magnetic Resonance Imaging (MRI) scans are performed with different contrast characteristics at a single patient visit. Unfortunately, MRI data-acquisition is inherently slow creating a persistent need to accelerate scans. Multi-contrast reconstruction deals with the joint reconstruction of different contrasts simultaneously. Previous approaches suggest solving a regularized optimization problem using group sparsity and/or color total variation, using composite-splitting denoising and FISTA. Yet, there is significant room for improvement in existing methods regarding computation time, ease of parameter selection, and robustness in reconstructed image quality. Selection of sparsifying transformations is critical in applications of compressed sensing. Here we propose using non-convex p-norm group sparsity (with p < 1), and apply color total variation (CTV). Our method is readily applicable to magnitude images rather than each of the real and imaginary parts separately. We use the constrained form of the problem, which allows an easier choice of data-fidelity error-bound (based on noise power determined from a noise-only scan without any RF excitation). We solve the problem using an adaptation of Alternating Direction Method of Multipliers (ADMM), which provides faster convergence in terms of CPU-time. We demonstrated the effectiveness of the method on two MR image sets (numerical brain phantom images and SRI24 atlas data) in terms of CPU-time and image quality. We show that a non-convex group sparsity function that uses the p-norm instead of the convex counterpart accelerates convergence and improves the peak-Signal-to-Noise-Ratio (pSNR), especially for highly undersampled data.

As the capability of high-resolution displays grows, high-resolution images are often required in Computed Tomography (CT). However, acquiring high-resolution images takes a higher radiation dose and a longer scanning time. In this study, we applied the Sparse-coding-based Super-Resolution (ScSR) method to generate high-resolution images without increasing the radiation dose. We prepared the over-complete dictionary learned the mapping between low- and highresolution patches and seek a sparse representation of each patch of the low-resolution input. These coefficients were used to generate the high-resolution output. For evaluation, 44 CT cases were used as the test dataset. We up-sampled images up to 2 or 4 times and compared the image quality of the ScSR scheme and bilinear and bicubic interpolations, which are the traditional interpolation schemes. We also compared the image quality of three learning datasets. A total of 45 CT images, 91 non-medical images, and 93 chest radiographs were used for dictionary preparation respectively. The image quality was evaluated by measuring peak signal-to-noise ratio (PSNR) and structure similarity (SSIM). The differences of PSNRs and SSIMs between the ScSR method and interpolation methods were statistically significant. Visual assessment confirmed that the ScSR method generated a high-resolution image with sharpness, whereas conventional interpolation methods generated over-smoothed images. To compare three different training datasets, there were no significance between the CT, the CXR and non-medical datasets. These results suggest that the ScSR provides a robust approach for application of up-sampling CT images and yields substantial high image quality of extended images in CT.

Motion correction of dynamic contrast enhanced magnetic resonance images (DCE-MRI) is a challenging task, due to changes in image appearance. In this study a groupwise registration, using a principle component analysis (PCA) based metric, is evaluated for clinical DCE MRI of the liver. The groupwise registration transforms the images to a common space, rather than to a reference volume as conventional pairwise methods do, and computes the similarity metric on all volumes simultaneously. This groupwise registration method is compared to a pairwise approach using a mutual information metric. Clinical DCE MRI of the abdomen of eight patients were included. Per patient one lesion in the liver was manually segmented in all temporal images (N=16). The registered images were compared for accuracy, spatial and temporal smoothness after transformation, and lesion volume change. Compared to a pairwise method or no registration, groupwise registration provided better alignment. In our recently started clinical study groupwise registered clinical DCE MRI of the abdomen of nine patients were scored by three radiologists. Groupwise registration increased the assessed quality of alignment. The gain in reading time for the radiologist was estimated to vary from no difference to almost a minute. A slight increase in reader confidence was also observed. Registration had no added value for images with little motion. In conclusion, the groupwise registration of DCE MR images results in better alignment than achieved by pairwise registration, which is beneficial for clinical assessment.

Error estimation in medical image registration is valuable when validating, comparing, or combining registration methods. To validate a nonlinear image registration method, ideally the registration error should be known for the entire image domain. We propose a supervised method for the estimation of a registration error map for nonlinear image registration. The method is based on a convolutional neural network that estimates the norm of the residual deformation from patches around each pixel in two registered images. This norm is interpreted as the registration error, and is defined for every pixel in the image domain. The network is trained using a set of artificially deformed images. Each training example is a pair of images: the original image, and a random deformation of that image. No manually labeled ground truth error is required. At test time, only the two registered images are required as input. We train and validate the network on registrations in a set of 2D digital subtraction angiography sequences, such that errors up to eight pixels can be estimated. We show that for this range of errors the convolutional network is able to learn the registration error in pairs of 2D registered images at subpixel precision. Finally, we present a proof of principle for the extension to 3D registration problems in chest CTs, showing that the method has the potential to estimate errors in 3D registration problems.

The Gleason grading system used to render prostate cancer diagnosis has recently been updated to allow more accurate grade stratification and higher prognostic discrimination when compared to the traditional grading system. In spite of progress made in trying to standardize the grading process, there still remains approximately a 30% grading discrepancy between the score rendered by general pathologists and those provided by experts while reviewing needle biopsies for Gleason pattern 3 and 4, which accounts for more than 70% of daily prostate tis- sue slides at most institutions. We propose a new computational imaging method for Gleason pattern 3 and 4 classification, which better matches the newly established prostate cancer grading system. The computer- aided analysis method includes two phases. First, the boundary of each glandular region is automatically segmented using a deep convolutional neural network. Second, color, shape and texture features are extracted from superpixels corresponding to the outer and inner glandular regions and are subsequently forwarded to a random forest classifier to give a gradient score between 3 and 4 for each delineated glandular region. The F1 score for glandular segmentation is 0.8460 and the classification accuracy is 0.83±0.03.

In developing treatment of cardiovascular diseases, short axis cine MRI has been used as a standard technique for understanding the global structural and functional characteristics of the heart, e.g. ventricle dimensions, stroke volume and ejection fraction. To conduct an accurate assessment, heart structures need to be segmented from the cine MRI images with high precision, which could be a laborious task when performed manually. Herein a fully automatic framework is proposed for the segmentation of the left ventricle from the slices of short axis cine MRI scans of porcine subjects using a deep learning approach. For training the deep learning models, which generally requires a large set of data, a public database of human cine MRI scans is used. Experiments on the 3150 cine slices of 7 porcine subjects have shown that when comparing the automatic and manual segmentations the mean slice-wise Dice coefficient is about 0.930, the point-to-curve error is 1.07 mm, and the mean slice-wise Hausdorff distance is around 3.70 mm, which demonstrates the accuracy and robustness of the proposed inter-species translational approach.

Accurate volume measurements of pituitary adenoma are important to the diagnosis and treatment for this kind of sellar tumor. The pituitary adenomas have different pathological representations and various shapes. Particularly, in the case of infiltrating to surrounding soft tissues, they present similar intensities and indistinct boundary in T1-weighted (T1W) magnetic resonance (MR) images. Then the extraction of pituitary adenoma from MR images is still a challenging task. In this paper, we propose an interactive method to segment the pituitary adenoma from brain MR data, by combining graph cuts based active contour model (GCACM) and random walk algorithm. By using the GCACM method, the segmentation task is formulated as an energy minimization problem by a hybrid active contour model (ACM), and then the problem is solved by the graph cuts method. The region-based term in the hybrid ACM considers the local image intensities as described by Gaussian distributions with different means and variances, expressed as maximum a posteriori probability (MAP). Random walk is utilized as an initialization tool to provide initialized surface for GCACM. The proposed method is evaluated on the three-dimensional (3-D) T1W MR data of 23 patients and compared with the standard graph cuts method, the random walk method, the hybrid ACM method, a GCACM method which considers global mean intensity in region forces, and a competitive region-growing based GrowCut method planted in 3D Slicer. Based on the experimental results, the proposed method is superior to those methods.

In this paper, we develop an improved auto-focusing capability of a panoramic dental tomosynthesis imager. We propose an auto-focusing algorithm with an efficient sharpness indicator based on exponential polynomials which provides better quantitation of steep gradients than the conventional one based on algebraic polynomials. With its accurate estimation of the sharpness of the reconstructed slices, the proposed method resulted in a better performance of automatically extracting in-focus slices in the dental panoramic tomosynthesis.

To reduce cupping artifacts and enhance contrast resolution in cone-beam CT (CBCT), in this paper, we introduce a new approach which combines blind deconvolution with a level set method. The proposed method focuses on the reconstructed image without requiring any additional physical equipment, is easily implemented on a single-scan acquisition. The results demonstrate that the algorithm is practical and effective for reducing the cupping artifacts and enhance contrast resolution on the images, preserves the quality of the reconstructed image, and is very robust.

Standard Single Photon Emission Computed Tomography (SPECT) has a limited field of view (FOV) and cannot provide a 3D image of an entire long whole body SPECT. To produce a 3D whole body SPECT image, two to five overlapped SPECT FOVs from head to foot are acquired and assembled using image stitching. Most commercial software from medical imaging manufacturers applies a direct mid-slice stitching method to avoid blurring or ghosting from 3D image blending. Due to intensity changes across the middle slice of overlapped images, direct mid-slice stitching often produces visible seams in the coronal and sagittal views and maximal intensity projection (MIP). In this study, we proposed an optimized algorithm to reduce the visibility of stitching edges. The new algorithm computed, based on transition error minimization (TEM), a 3D stitching interface between two overlapped 3D SPECT images. To test the suggested algorithm, four studies of 2-FOV whole body SPECT were used and included two different reconstruction methods (filtered back projection (FBP) and ordered subset expectation maximization (OSEM)) as well as two different radiopharmaceuticals (Tc-99m MDP for bone metastases and I-131 MIBG for neuroblastoma tumors). Relative transition errors of stitched whole body SPECT using mid-slice stitching and the TEM-based algorithm were measured for objective evaluation. Preliminary experiments showed that the new algorithm reduced the visibility of the stitching interface in the coronal, sagittal, and MIP views. Average relative transition errors were reduced from 56.7% of mid-slice stitching to 11.7% of TEM-based stitching. The proposed algorithm also avoids blurring artifacts by preserving the noise properties of the original SPECT images.

Visual assessment is the most common clinical investigation of skin reactions in radiotherapy. Due to the subjective nature of this method, additional noninvasive techniques are needed for more accurate evaluation. Our goal is to evaluate the effectiveness of hyperspectral image analysis for that purpose. In this pilot study, we focused on detection and grading of skin Erythema. This paper reports our proposed processing pipeline and experimental findings. Experiments have been performed to demonstrate the efficacy of the proposed approach for (1) reproducing clinical assessments, and (2) outperforming RGB imaging data.

Ischemic stroke has great correlation with carotid atherosclerosis and is mostly caused by vulnerable plaques. It’s particularly important to analysis the components of plaques for the detection of vulnerable plaques. Recently plaque analysis based on multi-contrast magnetic resonance imaging has attracted great attention. Though multi-contrast MR imaging has potentials in enhanced demonstration of carotid wall, its performance is hampered by the misalignment of different imaging sequences. In this study, a coarse-to-fine registration strategy based on cross-sectional images and wall boundaries is proposed to solve the problem. It includes two steps: a rigid step using the iterative closest points to register the centerlines of carotid artery extracted from multi-contrast MR images, and a non-rigid step using the thin plate spline to register the lumen boundaries of carotid artery. In the rigid step, the centerline was extracted by tracking the crosssectional images along the vessel direction calculated by Hessian matrix. In the non-rigid step, a shape context descriptor is introduced to find corresponding points of two similar boundaries. In addition, the deterministic annealing technique is used to find a globally optimized solution. The proposed strategy was evaluated by newly developed three-dimensional, fast and high resolution multi-contrast black blood MR imaging. Quantitative validation indicated that after registration, the overlap of two boundaries from different sequences is 95%, and their mean surface distance is 0.12 mm. In conclusion, the proposed algorithm has improved the accuracy of registration effectively for further component analysis of carotid plaques.

We have proposed an end-to-end learning approach that trained a deep convolutional neural network (CNN) for automatic CT image segmentation, which accomplished a voxel-wised multiple classification to directly map each voxel on 3D CT images to an anatomical label automatically. The novelties of our proposed method were (1) transforming the anatomical structures segmentation on 3D CT images into a majority voting of the results of 2D semantic image segmentation on a number of 2D-slices from different image orientations, and (2) using “convolution” and “deconvolution” networks to achieve the conventional “coarse recognition” and “fine extraction” functions which were integrated into a compact all-in-one deep CNN for CT image segmentation. The advantage comparing to previous works was its capability to accomplish real-time image segmentations on 2D slices of arbitrary CT-scan-range (e.g. body, chest, abdomen) and produced correspondingly-sized output. In this paper, we propose an improvement of our proposed approach by adding an organ localization module to limit CT image range for training and testing deep CNNs. A database consisting of 240 3D CT scans and a human annotated ground truth was used for training (228 cases) and testing (the remaining 12 cases). We applied the improved method to segment pancreas and left kidney regions, respectively. The preliminary results showed that the accuracies of the segmentation results were improved significantly (pancreas was 34% and kidney was 8% increased in Jaccard index from our previous results). The effectiveness and usefulness of proposed improvement for CT image segmentations were confirmed.

Microscopic image registration is a key component of the neural structure reconstruction with serial sections of neural tissue. The goal of microscopic neural image registration is to recover the 3D continuity and geometrical properties of specimen. During image registration, various distortions need to be corrected, including image rotation, translation, tissue deformation et.al, which come from the procedure of sample cutting, staining and imaging. Furthermore, there is only certain similarity between adjacent sections, and the degree of similarity depends on local structure of the tissue and the thickness of the sections. These factors make the microscopic neural image registration a challenging problem. To tackle the difficulty of corresponding landmarks extraction, we introduce a novel image registration method for Scanning Electron Microscopy (SEM) images of serial neural tissue sections based on the structure of mitochondria. The ellipsoidal shape of mitochondria ensures that the same mitochondria has similar shape between adjacent sections, and its characteristic of broad distribution in the neural tissue guarantees that landmarks based on the mitochondria distributed widely in the image. The proposed image registration method contains three parts: landmarks extraction between adjacent sections, corresponding landmarks matching and image deformation based on the correspondences. We demonstrate the performance of our method with SEM images of drosophila brain.

Automated face detection is the pivotal step in computer vision aided facial medical diagnosis and biometrics. This paper presents an automatic, subject adaptive framework for accurate face detection in the long infrared spectrum on our database for oral cancer detection consisting of malignant, precancerous and normal subjects of varied age group. Previous works on oral cancer detection using Digital Infrared Thermal Imaging(DITI) reveals that patients and normal subjects differ significantly in their facial thermal distribution. Therefore, it is a challenging task to formulate a completely adaptive framework to veraciously localize face from such a subject specific modality. Our model consists of first extracting the most probable facial regions by minimum error thresholding followed by ingenious adaptive methods to leverage the horizontal and vertical projections of the segmented thermal image. Additionally, the model incorporates our domain knowledge of exploiting temperature difference between strategic locations of the face. To our best knowledge, this is the pioneering work on detecting faces in thermal facial images comprising both patients and normal subjects. Previous works on face detection have not specifically targeted automated medical diagnosis; face bounding box returned by those algorithms are thus loose and not apt for further medical automation. Our algorithm significantly outperforms contemporary face detection algorithms in terms of commonly used metrics for evaluating face detection accuracy. Since our method has been tested on challenging dataset consisting of both patients and normal subjects of diverse age groups, it can be seamlessly adapted in any DITI guided facial healthcare or biometric applications.

Cortical bone plays a big role in the mechanical competence of bone. The analysis of cortical bone requires accurate segmentation methods. Level set methods are usually in the state-of-the-art for segmenting medical images. However, traditional implementations of this method are computationally expensive. This drawback was recently tackled through the so-called coherent propagation extension of the classical algorithm which has decreased computation times dramatically. In this study, we assess the potential of this technique for segmenting cortical bone in interactive time in 3D images acquired through High Resolution peripheral Quantitative Computed Tomography (HR-pQCT). The obtained segmentations are used to estimate cortical thickness and cortical porosity of the investigated images. Cortical thickness and Cortical porosity is computed using sphere fitting and mathematical morphological operations respectively. Qualitative comparison between the segmentations of our proposed algorithm and a previously published approach on six images volumes reveals superior smoothness properties of the level set approach. While the proposed method yields similar results to previous approaches in regions where the boundary between trabecular and cortical bone is well defined, it yields more stable segmentations in challenging regions. This results in more stable estimation of parameters of cortical bone. The proposed technique takes few seconds to compute, which makes it suitable for clinical settings.

Spare-view sampling and its associated iterative image reconstruction in computed tomography have actively investigated. Sparse-view CT technique is a viable option to low-dose CT, particularly in cone-beam CT (CBCT) applications, with advanced iterative image reconstructions with varying degrees of image artifacts. One of the artifacts that may occur in sparse-view CT is the streak artifact in the reconstructed images. Another approach has been investigated for sparse-view CT imaging by use of the interpolation methods to fill in the missing view data and that reconstructs the image by an analytic reconstruction algorithm. In this study, we developed an interpolation method using convolutional neural network (CNN), which is one of the widely used deep-learning methods, to find missing projection data and compared its performances with the other interpolation techniques.

Extracting the structure of single neurons is critical for understanding how they function within the neural circuits. Recent developments in microscopy techniques, and the widely recognized need for openness and standardization provide a community resource for automated reconstruction of dendritic and axonal morphology of single neurons. In order to look into the fine structure of neurons, we use the Automated Tape-collecting Ultra Microtome Scanning Electron Microscopy (ATUM-SEM) to get images sequence of serial sections of animal brain tissue that densely packed with neurons. Different from other neuron reconstruction method, we propose a method that enhances the SEM images by detecting the neuronal membranes with deep convolutional neural network (DCNN) and segments single neurons by active contour with group shape similarity. We joint the segmentation and tracing together and they interact with each other by alternate iteration that tracing aids the selection of candidate region patch for active contour segmentation while the segmentation provides the neuron geometrical features which improve the robustness of tracing. The tracing model mainly relies on the neuron geometrical features and is updated after neuron being segmented on the every next section. Our method enables the reconstruction of neurons of the drosophila mushroom body which is cut to serial sections and imaged under SEM. Our method provides an elementary step for the whole reconstruction of neuronal networks.

Similarity learning is one of the most fundamental tasks in image analysis. The ability to extract similar images in the medical domain as part of content-based image retrieval (CBIR) systems has been researched for many years. The vast majority of methods used in CBIR systems are based on hand-crafted feature descriptors. The approximation of a similarity mapping for medical images is difficult due to the big variety of pixel-level structures of interest. In fundus photography (FP) analysis, a subtle difference in e.g. lesions and vessels shape and size can result in a different diagnosis. In this work, we demonstrated how to learn a similarity function for image patches derived directly from FP image data without the need of manually designed feature descriptors. We used a convolutional neural network (CNN) with a novel architecture adapted for similarity learning to accomplish this task. Furthermore, we explored and studied multiple CNN architectures. We show that our method can approximate the similarity between FP patches more efficiently and accurately than the state-of- the-art feature descriptors, including SIFT and SURF using a publicly available dataset. Finally, we observe that our approach, which is purely data-driven, learns that features such as vessels calibre and orientation are important discriminative factors, which resembles the way how humans reason about similarity. To the best of authors knowledge, this is the first attempt to approximate a visual similarity mapping in FP.

Both traditional and wireless capsule endoscopes can generate tens of thousands of images for each patient. It is desirable to have the majority of irrelevant images filtered out by automatic algorithms during an offline review process or to have automatic indication for highly suspicious areas during an online guidance. This also applies to the newly invented endomicroscopy, where online indication of tumor classification plays a significant role. Image classification is a standard pattern recognition problem and is well studied in the literature. However, performance on the challenging endoscopic images still has room for improvement. In this paper, we present a novel Cascaded Deep Decision Network (CDDN) to improve image classification performance over standard Deep neural network based methods. During the learning phase, CDDN automatically builds a network which discards samples that are classified with high confidence scores by a previously trained network and concentrates only on the challenging samples which would be handled by the subsequent expert shallow networks. We validate CDDN using two different types of endoscopic imaging, which includes a polyp classification dataset and a tumor classification dataset. From both datasets we show that CDDN can outperform other methods by about 10%. In addition, CDDN can also be applied to other image classification problems.

Endovascular aneurysm repair (EVAR) of juxtarenal aortic aneurysms (JAA) is particularly challenging owing to the requirement of suprarenal EVAR graft fixation, which has been associated with significant declines in long term renal function. Therefore, the ability to design fenestrated EVAR grafts on a personalized basis in order to ensure visceral and renal perfusion, is highly desirable. The objectives of this study are: a) To demonstrate novel 3D geometric methods to virtually design and deploy EVAR grafts into a virtually designed JAA, by applying a custom surface mesh deformation tool to a patient-specific descending aortic model reconstructed from computed tomographic (CT) images; and b) To virtually evaluate patient-specific renal flow and wall stresses in these patient-specific virtually EVAR geometries, using computational fluid dynamics (CFD). The presented framework may provide the modern cardiovascular surgeon the ability to leverage non-invasive, pre-operative imaging equipment to personalize and guide EVAR therapeutic strategy. Our CFD studies revealed that virtual EVAR grafting of a patient-specific JAA, with optimal fenestration sites and renal stenting, led to a 179.67±15.95% and 1051.43±18.34% improvement in right and left renal flow rates, respectively, when compared with the baseline patient-specific aortic geometry with renal stenoses, whereas a right and left renal flow improved by 36.44±2.24% and 885.93±12.41%, respectively, relative to the equivalently modeled JAA with renal stenoses, considering averages across the three simulated inflow rate cases. The proposed framework have utility to iteratively optimize suprarenal EVAR fixation length and achieve normal renal wall shear stresses and streamlined juxtarenal hemodynamics.

Known for its distinct role in memory, the hippocampus is one of the most studied regions of the brain. Recent advances in magnetic resonance imaging have allowed for high-contrast, reproducible imaging of the hippocampus. Typically, a trained rater takes 45 minutes to manually trace the hippocampus and delineate the anterior from the posterior segment at millimeter resolution. As a result, there has been a significant desire for automated and robust segmentation of the hippocampus. In this work we use a population of 195 atlases based on T1-weighted MR images with the left and right hippocampus delineated into the head and body. We initialize the multi-atlas segmentation to a region directly around each lateralized hippocampus to both speed up and improve the accuracy of registration. This initialization allows for incorporation of nearly 200 atlases, an accomplishment which would typically involve hundreds of hours of computation per target image. The proposed segmentation results in a Dice similiarity coefficient over 0.9 for the full hippocampus. This result outperforms a multi-atlas segmentation using the BrainCOLOR atlases (Dice 0.85) and FreeSurfer (Dice 0.75). Furthermore, the head and body delineation resulted in a Dice coefficient over 0.87 for both structures. The head and body volume measurements also show high reproducibility on the Kirby 21 reproducibility population (R² greater than 0.95, p < 0.05 for all structures). This work signifies the first result in an ongoing work to develop a robust tool for measurement of the hippocampus and other temporal lobe structures.

Corneal topography is a medical imaging technique to get the 3D shape of the cornea as a set of 3D points of its anterior and posterior surfaces. From these data, topographic maps can be derived to assist the ophthalmologist in the diagnosis of disorders. In this paper, we compare three different mathematical parametric representations of the corneal surfaces leastsquares fitted to the data provided by corneal topography. The parameters obtained from these models reduce the dimensionality of the data from several thousand 3D points to only a few parameters and could eventually be useful for diagnosis, biometry, implant design etc. The first representation is based on Zernike polynomials that are commonly used in optics. A variant of these polynomials, named Bhatia-Wolf will also be investigated. These two sets of polynomials are defined over a circular domain which is convenient to model the elevation (height) of the corneal surface. The third representation uses Spherical Harmonics that are particularly well suited for nearly-spherical object modeling, which is the case for cornea. We compared the three methods using the following three criteria: the root-mean-square error (RMSE), the number of parameters and the visual accuracy of the reconstructed topographic maps. A large dataset of more than 2000 corneal topographies was used. Our results showed that Spherical Harmonics were superior with a RMSE mean lower than 2.5 microns with 36 coefficients (order 5) for normal corneas and lower than 5 microns for two diseases affecting the corneal shapes: keratoconus and Fuchs’ dystrophy.

Dynamic contrast-enhanced computed tomography (DCE-CT) is an emerging radiological technique, which consists in acquiring a rapid sequence of CT images, shortly after the injection of an intravenous contrast agent. The passage of the contrast agent in a tissue results in a varying CT intensity over time, recorded in time-attenuation curves (TACs), which can be related to the contrast supplied to that tissue via the supplying artery to estimate the local perfusion and permeability characteristics. The time delay between the arrival of the contrast bolus in the feeding artery and the tissue of interest, called the bolus arrival time (BAT), needs to be determined accurately to enable reliable perfusion analysis. Its automated identification is however highly sensitive to noise. We propose an accurate and efficient method for estimating the BAT from DCE-CT images. The method relies on a piecewise linear TAC model with four segments and suitable parameter constraints for limiting the range of possible values. The model is fitted to the acquired TACs in a multiresolution fashion using an iterative optimization approach. The performance of the method was evaluated on simulated and real perfusion data of lung and rectum tumours. In both cases, the method was found to be stable, leading to average accuracies in the order of the temporal resolution of the dynamic sequence. For reasonable levels of noise, the results were found to be comparable to those obtained using a previously proposed method, employing a full search algorithm, but requiring an order of magnitude more computation time.

Statistical Atlases have played an important role towards automated medical image segmentation. However, a challenge has been to make the atlas more adaptable to possible errors in deformable registration of anomalous images, given that the body structures of interest for segmentation might present significant differences in shape and texture. Recently, deformable registration errors have been accounted by a method that locally translates the statistical atlas over the test image, after registration, and evaluates candidate objects from a delineation algorithm in order to choose the best one as final segmentation. In this paper, we improve its delineation algorithm and extend the model to be a multi-object statistical atlas, built from control images and adaptable to anomalous images, by incorporating a texture classifier. In order to provide a first proof of concept, we instantiate the new method for segmenting, object-by-object and all objects simultaneously, the left and right brain hemispheres, and the cerebellum, without the brainstem, and evaluate it on MRT1-images of epilepsy patients before and after brain surgery, which removed portions of the temporal lobe. The results show efficiency gain with statistically significant higher accuracy, using the mean Average Symmetric Surface Distance, with respect to the original approach.

Accurate automatic prostate magnetic resonance image (MRI) segmentation is a challenging task due to the high variability of prostate anatomic structure. Artifacts such as noise and similar signal intensity tissues around the prostate boundary inhibit traditional segmentation methods from achieving high accuracy. The proposed method performs end-to- end segmentation by integrating holistically nested edge detection with fully convolutional neural networks. Holistically-nested networks (HNN) automatically learn the hierarchical representation that can improve prostate boundary detection. Quantitative evaluation is performed on the MRI scans of 247 patients in 5-fold cross-validation. We achieve a mean Dice Similarity Coefficient of 88.70% and a mean Jaccard Similarity Coefficient of 80.29% without trimming any erroneous contours at apex and base.

Optical coherence tomography (OCT) provides high-resolution cross-sectional images of arterial luminal morphology. Traditionally lumen segmentation of OCT images is performed manually by expert observers; a laborious, time consuming effort, sensitive to inter-observer variability process. Although several automated methods have been developed, the majority cannot be applied in real time because of processing demands. To address these limitations we propose a new method for rapid image segmentation of arterial lumen borders using OCT images that involves the following steps: 1) OCT image acquisition using the raw OCT data, 2) reconstruction of longitudinal cross-section (LOCS) images from four different acquisition angles, 3) segmentation of the LOCS images and 4) lumen contour construction in each 2D cross-sectional image. The efficiency of the developed method was evaluated using 613 annotated images from 10 OCT pullbacks acquired from 10 patients at the time of coronary arterial interventions. High Pearson’s correlation coefficient was obtained when lumen areas detected by the method were compared to areas annotated by experts (r=0.98, R²=0.96); Bland-Altman analysis showed no significant bias with good limits of agreement. The proposed methodology permits reliable border detection especially in lumen areas having artifacts and is faster than traditional techniques making it capable of being used in real time applications. The method is likely to assist in a number of research and clinical applications - further testing in an expanded clinical arena will more fully define the limits and potential of this approach.

Interpretation of temporal CT images could help the radiologists to detect some subtle interval changes in the sequential examinations. The purpose of this study was to develop a fully automated scheme for accurate registration of temporal CT images for pulmonary nodule detection. Our method consisted of three major registration steps. Firstly, affine transformation was applied in the segmented lung region to obtain global coarse registration images. Secondly, B-splines based free-form deformation (FFD) was used to refine the coarse registration images. Thirdly, Demons algorithm was performed to align the feature points extracted from the registered images in the second step and the reference images. Our database consisted of 91 temporal CT cases obtained from Beijing 301 Hospital and Shanghai Changzheng Hospital. The preliminary results showed that approximately 96.7% cases could obtain accurate registration based on subjective observation. The subtraction images of the reference images and the rigid and non-rigid registered images could effectively remove the normal structures (i.e. blood vessels) and retain the abnormalities (i.e. pulmonary nodules). This would be useful for the screening of lung cancer in our future study.

Colon cancer is the second cancer killer in the US [1]. Colonoscopy is the primary method for screening and prevention of colon cancer, but during colonoscopy, a significant number (25% [2]) of polyps (precancerous abnormal growths inside of the colon) are missed; therefore, the goal of our research is to reduce the polyp miss-rate of colonoscopy. This paper presents a method to detect polyp automatically in a colonoscopy video. Our system has two stages: Candidate generation and candidate classification. In candidate generation (stage 1), we chose 3,463 frames (including 1,718 with-polyp frames) from real-time colonoscopy video database. We first applied processing procedures, namely intensity adjustment, edge detection and morphology operations, as pre-preparation. We extracted each connected component (edge contour) as one candidate patch from the pre-processed image. With the help of ground truth (GT) images, 2 constraints were implemented on each candidate patch, dividing and saving them into polyp group and non-polyp group. In candidate classification (stage 2), we trained and tested convolutional neural networks (CNNs) with AlexNet architecture [3] to classify each candidate into with-polyp or non-polyp class. Each with-polyp patch was processed by rotation, translation and scaling for invariant to get a much robust CNNs system. We applied leave-2-patients-out cross-validation on this model (4 of 6 cases were chosen as training set and the rest 2 were as testing set). The system accuracy and sensitivity are 91.47% and 91.76%, respectively.

Ultrasound imaging is widely used in clinical imaging because it is non-invasive, real-time, and inexpensive. Due to the freehand nature of clinical ultrasound, analysis of an image sequence often requires registration between the images. Of the previously developed mono-modality ultrasound registration frameworks, only few were designed to register small anatomical structures. Monitoring of small finger vessels, in particular, is essential for the treatment of vascular diseases such as Raynaud’s Disease. High frequency ultrasound (HFUS) can now image smaller anatomic details down to 30 microns within the vessels, but no work has been done to date on such small-scale ultrasound registration. Due to the complex internal finger structure and increased noise of HFUS, it is difficult to register 2D images of finger vascular tissue, especially under deformation. We studied a variety of similarity measurements with different pre-processing techniques to find which registration similarity metrics were best suited for HFUS vessel tracking. The overall best performance was obtained with a normalized correlation metric coupled with HFUS downsampling and a one-plus-one evolutionary optimizer, yielding a mean registration error of 0.05 mm. We also used HFUS to study how finger tissue deforms under an ultrasound transducer, comparing internal motion vs. transducer motion. Improving HFUS registration and tissue modeling may lead to new research and improved treatments for peripheral vascular disorders.

Medical image registration establishes a correspondence between images of biological structures and it is at the core of many applications. Commonly used deformable image registration methods are dependent on a good preregistration initialization. The initialization can be performed by localizing homologous landmarks and calculating a point-based transformation between the images. The selection of landmarks is however important. In this work, we present a learning-based method to automatically find a set of robust landmarks in 3D MR image volumes of the head to initialize non-rigid transformations. To validate our method, these selected landmarks are localized in unknown image volumes and they are used to compute a smoothing thin-plate splines transformation that registers the atlas to the volumes. The transformed atlas image is then used as the preregistration initialization of an intensity-based non-rigid registration algorithm. We show that the registration accuracy of this algorithm is statistically significantly improved when using the presented registration initialization over a standard intensity-based affine registration.

After more than 25 years of published functional magnetic resonance imaging (fMRI) studies, careful scrutiny reveals that most of the reported results lack fully decisive validation. The complex nature of fMRI data generation and acquisition results in unavoidable uncertainties in the true estimation and interpretation of both task-related activation maps and resting state functional connectivity networks, despite the use of various statistical data analysis methodologies. The goal of developing the proposed STANCE (Spontaneous and Task-related Activation of Neuronally Correlated Events) simulator is to generate realistic task-related and/or resting-state 4D blood oxygenation level dependent (BOLD) signals, given the experimental paradigm and scan protocol, by using digital phantoms of twenty normal brains available from BrainWeb (http://brainweb.bic.mni.mcgill.ca/brainweb/). The proposed simulator will include estimated system and modelled physiological noise as well as motion to serve as a reference to measured brain activities. In its current form, STANCE is a MATLAB toolbox with command line functions serving as an open-source add-on to SPM8 (http://www.fil.ion.ucl.ac.uk/spm/software/spm8/). The STANCE simulator has been designed in a modular framework so that the hemodynamic response (HR) and various noise models can be iteratively improved to include evolving knowledge about such models.

Automatic segmentation of the prostate on CT images has many applications in prostate cancer diagnosis and therapy. However, prostate CT image segmentation is challenging because of the low contrast of soft tissue on CT images. In this paper, we propose an automatic segmentation method by combining a deep learning method and multi-atlas refinement. First, instead of segmenting the whole image, we extract the region of interesting (ROI) to delete irrelevant regions. Then, we use the convolutional neural networks (CNN) to learn the deep features for distinguishing the prostate pixels from the non-prostate pixels in order to obtain the preliminary segmentation results. CNN can automatically learn the deep features adapting to the data, which are different from some handcrafted features. Finally, we select some similar atlases to refine the initial segmentation results. The proposed method has been evaluated on a dataset of 92 prostate CT images. Experimental results show that our method achieved a Dice similarity coefficient of 86.80% as compared to the manual segmentation. The deep learning based method can provide a useful tool for automatic segmentation of the prostate on CT images and thus can have a variety of clinical applications.

Image matting is a method that separates foreground and background objects in an image, and has been widely used in medical image segmentation. Previous work has shown that matting can be formulated as a graph Laplacian matrix. In this paper, we derived matting from a local regression and global alignment view, as an attempt to provide a more intuitive solution to the segmentation problem. In addition, we improved the matting algorithm by adding a weight extension and refer to the proposed approach as Adaptive Weight Matting (AWM), where an adaptive weight was added to each local regression term to reduce the bias caused by outliers. We compared the segmentation results generated by the proposed method and several state-of-the-art segmentation methods, including conventional matting, graph-cuts and random walker, on medical images of different organs acquired using different imaging modalities. Experimental results demonstrated the advantages of AWM on medical image segmentation.

Recently, MR simulators gain popularity because of unnecessary radiation exposure of CT simulators being used in radiation therapy planning. We propose a method for pseudo CT estimation from MR images based on a patch-based random forest. Patient-specific anatomical features are extracted from the aligned training images and adopted as signatures for each voxel. The most robust and informative features are identified using feature selection to train the random forest. The well-trained random forest is used to predict the pseudo CT of a new patient. This prediction technique was tested with human brain images and the prediction accuracy was assessed using the original CT images. Peak signal-to-noise ratio (PSNR) and feature similarity (FSIM) indexes were used to quantify the differences between the pseudo and original CT images. The experimental results showed the proposed method could accurately generate pseudo CT images from MR images. In summary, we have developed a new pseudo CT prediction method based on patch-based random forest, demonstrated its clinical feasibility, and validated its prediction accuracy. This pseudo CT prediction technique could be a useful tool for MRI-based radiation treatment planning and attenuation correction in a PET/MRI scanner.

We present a method for semi-automatic segmentation of lung nodules in chest CT that can be extended to general lesion segmentation in multiple modalities. Most semi-automatic algorithms for lesion segmentation or similar tasks use region-growing or edge-based contour finding methods such as level-set. However, lung nodules and other lesions are often connected to surrounding tissues, which makes these algorithms prone to growing the nodule boundary into the surrounding tissue. To solve this problem, we apply a 3D extension of the 2D edge linking method with dynamic programming to find a closed surface in a spherical representation of the nodule ROI. The algorithm requires a user to draw a maximal diameter across the nodule in the slice in which the nodule cross section is the largest. We report the lesion volume estimation accuracy of our algorithm on the FDA lung phantom dataset, and the RECIST diameter estimation accuracy on the lung nodule dataset from the SPIE 2016 lung nodule classification challenge. The phantom results in particular demonstrate that our algorithm has the potential to mitigate the disparity in measurements performed by different radiologists on the same lesions, which could improve the accuracy of disease progression tracking.

The maximum mean cycle weight (MMCW) segmentation framework is a graph-based alternative to approaches such as GraphCut or Markov Random Fields. It offers time- and space-efficient computation and guaranteed optimality. However, unlike GraphCut or Markov Random Fields, MMCW does not seek to segment the entire image, but rather to find the single best object within the image, according to an objective function encoded by edge weights. Its focus on a single, best object makes MMCW attractive to interactive segmentation settings, where the user indicates which objects are to be segmented. However, a provably correct way of performing interactive segmentation using the MMCW framework has never been established. Further, the question of how to develop a good objective function based on user-provided information has never been addressed. Here, we propose a three-component objective function specifically designed for use with interactive MMCW segmentation. Two of those components, representing object boundary and object interior information, can be learned from a modest amount of user-labelled data, but in a way unique to the MMCW framework. The third component allows us to extend the MMCW framework to the situation of interactive segmentation. Specifically, we show that an appropriate weighted combination of the three components guarantees that the object produced by MMCW segmentation will enclose user-specified pixels that can be chosen interactively. The component weights can either be computed a priori based on image characteristics, or online via an adaptive reweighting scheme. We demonstrate the success of the approach on several microscope image segmentation problems.

Adipose tissue has been associated with adverse consequences of obesity. Total adipose tissue (TAT) is divided into subcutaneous adipose tissue (SAT) and visceral adipose tissue (VAT). Intra-abdominal fat (VAT), located inside the abdominal cavity, is a major factor for the classic obesity related pathologies. Since direct measurement of visceral and subcutaneous fat is not trivial, substitute metrics like waist circumference (WC) and body mass index (BMI) are used in clinical settings to quantify obesity. Abdominal fat can be assessed effectively using CT or MRI, but manual fat segmentation is rather subjective and time-consuming. Hence, an automatic and accurate quantification tool for abdominal fat is needed. The goal of this study is to extract TAT, VAT and SAT fat from abdominal CT in a fully automated unsupervised fashion using energy minimization techniques. We applied a four step framework consisting of 1) initial body contour estimation, 2) approximation of the body contour, 3) estimation of inner abdominal contour using Greedy Snakes algorithm, and 4) voting, to segment the subcutaneous and visceral fat. We validated our algorithm on 952 clinical abdominal CT images (from 476 patients with a very wide BMI range) collected from various radiology departments of Geisinger Health System. To our knowledge, this is the first study of its kind on such a large and diverse clinical dataset. Our algorithm obtained a 3.4% error for VAT segmentation compared to manual segmentation. These personalized and accurate measurements of fat can complement traditional population health driven obesity metrics such as BMI and WC.