Share Email Print

Proceedings Paper

Handling label noise through model confidence and uncertainty: application to chest radiograph classification
Author(s): Erdi Calli; Ecem Sogancioglu; Ernst Th. Scholten; Keelin Murphy; Bram van Ginneken
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

In this work we analyze the effect of label noise in training and test data when performing classification experiments on chest radiographs (CXRs) with modern deep learning architectures. We use ChestXRay14, the largest publicly available CXR dataset. We simulate situs inversus by horizontal flipping of the CXRs, allowing us to precisely control the amount of label noise. We also perform experiments in classifying emphysema using the ChestXRay14 provided labels that are known to be noisy. Our situs inversus experiments confirm results from the computer vision literature that deep learning architectures are relatively robust but not completely insensitive to label noise in the training data: without or with very low noise, classification results are near perfect; 16% and 32% training label noise only lead to a 1.5% and 4.6% drop in accuracy. We investigate two metrics that could be used to identify test samples that have an incorrect label: model confidence and model uncertainty. We show, in an observer study with an experienced chest radiologist, that both measures are effective in identifying samples in ChestXRay14 that are erroneously labeled for the presence of emphysema.

Paper Details

Date Published: 13 March 2019
PDF: 8 pages
Proc. SPIE 10950, Medical Imaging 2019: Computer-Aided Diagnosis, 1095016 (13 March 2019); doi: 10.1117/12.2514290
Show Author Affiliations
Erdi Calli, Diagnostic Image Analysis Group (Netherlands)
Ecem Sogancioglu, Diagnostic Image Analysis Group (Netherlands)
Ernst Th. Scholten, Diagnostic Image Analysis Group (Netherlands)
Keelin Murphy, Diagnostic Image Analysis Group (Netherlands)
Bram van Ginneken, Diagnostic Image Analysis Group (Netherlands)

Published in SPIE Proceedings Vol. 10950:
Medical Imaging 2019: Computer-Aided Diagnosis
Kensaku Mori; Horst K. Hahn, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?