Share Email Print

Proceedings Paper

Optimality of a utility-based performance metric for ROC analysis
Author(s): Darrin C. Edwards; Charles E. Metz
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

We previously introduced a utility-based ROC performance metric, the "surface-averaged expected cost" (SAEC), to address difficulties which arise in generalizing the well-known area under the ROC curve (AUC) to classification tasks with more than two classes. In a two-class classification task, the SAEC can be shown explicitly to be twice the area above the conventional ROC curve (1-AUC) divided by the arclength along the ROC curve. In the present work, we show that for a variety of two-class tasks under the binormal model, the SAEC obtained for the proper decision variable (the likelihood ratio of the latent decision variable) is less than that obtained for the conventional decision variable (i.e., using the latent decision variable directly). We also justify this result using a readily derived property of the arclength along the ROC curve under a given data model. Numerical studies as well as theoretical analysis suggest that the behavior of the SAEC is consistent with that of the AUC performance metric, in the sense that the optimal value of this quantity is achieved by the ideal observer.

Paper Details

Date Published: 6 March 2008
PDF: 6 pages
Proc. SPIE 6917, Medical Imaging 2008: Image Perception, Observer Performance, and Technology Assessment, 69170F (6 March 2008); doi: 10.1117/12.773052
Show Author Affiliations
Darrin C. Edwards, The Univ. of Chicago (United States)
Charles E. Metz, The Univ. of Chicago (United States)

Published in SPIE Proceedings Vol. 6917:
Medical Imaging 2008: Image Perception, Observer Performance, and Technology Assessment
Berkman Sahiner; David J. Manning, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?