Share Email Print
cover

Proceedings Paper

An exploration of uncertainty information for segmentation quality assessment
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Including uncertainty information in the assessment of a segmentation of pathologic structures on medical images, offers the potential to increase trust into deep learning algorithms for the analysis of medical imaging. Here, we examine options to extract uncertainty information from deep learning segmentation models and the influence of the choice of cost functions on these uncertainty measures. To this end we train conventional UNets without dropout, deep UNet ensembles, and Monte-Carlo (MC) dropout UNets to segment lung nodules on low dose CT using either soft Dice or weighted categorical cross-entropy (wcc) as loss functions. We extract voxel-wise uncertainty information from UNet models based on softmax maximum probability and from deep ensembles and MC dropout UNets using mean voxel-wise entropy. Upon visual assessment, areas of high uncertainty are localized in the periphery of segmentations and are in good agreement with incorrectly labelled voxels. Furthermore, we evaluate how well uncertainty measures correlate with segmentation quality (Dice score). Mean uncertainty over the segmented region (Ulabelled) derived from conventional UNet models does not show a strong quantitative relationship with the Dice score (Spearman correlation coefficient of -0.45 for the soft Dice vs -0.64 for the wcc model respectively). By comparison, image-level uncertainty measures derived from soft Dice as well as wcc MC UNet and deep UNet ensemble models correlate well with the Dice score. In conclusion, using uncertainty information offers ways to assess segmentation quality fully automatically without access to ground truth. Models trained using weighted categorical cross-entropy offer more meaningful uncertainty information on a voxel-level.

Paper Details

Date Published: 10 March 2020
PDF: 10 pages
Proc. SPIE 11313, Medical Imaging 2020: Image Processing, 113131K (10 March 2020);
Show Author Affiliations
Katharina Hoebel, Athinoula A. Martinos Ctr. for Biomedical Imaging (United States)
Harvard-MIT Division of Health Sciences and Technology (United States)
Vincent Andrearczyk, Univ. of Applied Sciences Western Switzerland (Switzerland)
Andrew Beers, Athinoula A. Martinos Ctr. for Biomedical Imaging (United States)
Jay Patel, Athinoula A. Martinos Ctr. for Biomedical Imaging (United States)
Harvard-MIT Division of Health Sciences and Technology (United States)
Ken Chang, Athinoula A. Martinos Ctr. for Biomedical Imaging (United States)
Harvard-MIT Division of Health Sciences and Technology (United States)
Adrien Depeursinge, Univ. of Applied Sciences Western Switzerland (Switzerland)
Ctr. Hospitalier Univ. Vaudois (Switzerland)
Henning Müller, Univ. of Applied Sciences Western Switzerland (Switzerland)
Univ. of Geneva (Switzerland)
Jayashree Kalpathy-Cramer, Athinoula A. Martinos Ctr. for Biomedical Imaging (United States)


Published in SPIE Proceedings Vol. 11313:
Medical Imaging 2020: Image Processing
Ivana Išgum; Bennett A. Landman, Editor(s)

© SPIE. Terms of Use
Back to Top
PREMIUM CONTENT
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?
close_icon_gray