Share Email Print

Proceedings Paper

Hypervolume under the ROC hypersurface of a near-guessing ideal observer in a three-class classification task
Author(s): Darrin C. Edwards; Charles E. Metz; Robert M. Nishikawa
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

We expressed the performance of the three-class "guessing" observer in terms of the six probabilities which make up a three-class receiver operating characteristic (ROC) space, in a formulation in which "sensitivities" are eliminated in constructing the ROC space (equivalent to using false-negative fraction and false-positive fraction in a two-class task). We then show that the "guessing" observer's performance in terms of these conditional probabilities is completely described by a degenerate hypersurface with only two degrees of freedom (as opposed to the five required, in general, to achieve a true hypersurface in such a ROC space). It readily follows that the hypervolume under such a degenerate hypersurface must be zero. We then consider a "near-guessing" task; that is, a task in which the three underlying data probability density functions (PDFs) are nearly identical, controlled by two parameters which may vary continuously to zero (at which point the PDFs become identical). The hypervolume under the ROC hypersurface of an observer in the three-class classification task tends continuously to zero as the underlying data PDFs converge continuously to identity (a "guessing" task). The hypervolume under the ROC hypersurface of a "perfect" ideal observer (a task in which the three data PDFs never overlap) is also found to be zero in the ROC space formulation under consideration. This suggests that hypervolume may not be a useful performance metric in three-class classification tasks, despite the utility of the area under the ROC curve for two-class tasks.

Paper Details

Date Published: 4 May 2004
PDF: 10 pages
Proc. SPIE 5372, Medical Imaging 2004: Image Perception, Observer Performance, and Technology Assessment, (4 May 2004); doi: 10.1117/12.536068
Show Author Affiliations
Darrin C. Edwards, Univ. of Chicago (United States)
Charles E. Metz, Univ. of Chicago (United States)
Robert M. Nishikawa, Univ. of Chicago (United States)

Published in SPIE Proceedings Vol. 5372:
Medical Imaging 2004: Image Perception, Observer Performance, and Technology Assessment
Dev P. Chakraborty; Miguel P. Eckstein, Editor(s)

© SPIE. Terms of Use
Back to Top