Share Email Print
cover

Proceedings Paper

Comparison of two classifiers when the data sets are imbalanced: the power of the area under the precision-recall curve as the figure of merit versus the area under the ROC curve
Author(s): Berkman Sahiner; Weijie Chen; Aria Pezeshk; Nicholas Petrick
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

In many two-class problems in automated classification and information retrieval, the classes are imbalanced, and the separation between the positive and negative classes is large. The precision-recall (PR) curve has been suggested as an alternative to the receiver operating characteristic (ROC) curve to characterize the performance of automated systems when the classes are imbalanced, and the area under the precision-recall curve (AUCPR) has been suggested as an alternative performance measure to the area under the ROC curve (AUCROC). AUCPR and AUCROC are distinct measures of performance, even though the relationship between the precision-recall and ROC curves is well-known. In this study, we compared the statistical power of the AUCPR to that of the AUCROC. Our results indicate that the AUCPR can offer a small statistical advantage when the prevalence is low and the separation between the positive and negative classes is large. When the data set is more balanced or the separation between the classes is low or moderate, AUCROC has slightly higher power.

Paper Details

Date Published: 10 March 2017
PDF: 9 pages
Proc. SPIE 10136, Medical Imaging 2017: Image Perception, Observer Performance, and Technology Assessment, 101360G (10 March 2017);
Show Author Affiliations
Berkman Sahiner, U.S. Food and Drug Administration (United States)
Weijie Chen, U.S. Food and Drug Administration (United States)
Aria Pezeshk, U.S. Food and Drug Administration (United States)
Nicholas Petrick, U.S. Food and Drug Administration (United States)


Published in SPIE Proceedings Vol. 10136:
Medical Imaging 2017: Image Perception, Observer Performance, and Technology Assessment
Matthew A. Kupinski; Robert M. Nishikawa, Editor(s)

© SPIE. Terms of Use
Back to Top
PREMIUM CONTENT
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?
close_icon_gray