Share Email Print

Proceedings Paper

ROC components-of-variance analysis of competing classifiers
Author(s): Sergey V. Beiden; Marcus A. Maloof; Robert F. Wagner
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

In the last decade in the field of diagnostic imaging, the problem of variability of reader skill as well as patient case difficulty has given rise to a multivariate approach to receiver operating characteristic (ROC) analysis. The multivariate approach is the so-called multiple-reader, multiple-case (MRMC) ROC paradigm in which every reader reads every patient case and, where possible, in each of two modalities under comparison. The present paper demonstrates the isomorphism between patient cases, image readers, and imaging modalities in diagnostic imaging and, respectively, test sets, training sets, and competing discriminant algorithms in the field of statistical pattern recognition (SPR). Thus, the MRMC paradigm can be brought directly across from imaging to SPR. Recent MRMC ROC analytical methods are demonstrated in the context of SPR for the task of analyzing the natural components of variance in that problem involving test sets, training sets, and competing discriminants. Monte Carlo trials reported here indicate that the conventional wisdom that the variance of measures of classifier accuracy comes mainly from the finite test set is only true when assessing a single algorithm in a very limited context. In particular, it is generally not true when comparing competing discriminant algorithms; in that case the variance is dominated by the finite training set.

Paper Details

Date Published: 9 May 2002
PDF: 10 pages
Proc. SPIE 4684, Medical Imaging 2002: Image Processing, (9 May 2002); doi: 10.1117/12.467176
Show Author Affiliations
Sergey V. Beiden, FDA Ctr. for Devices and Radiological Health (United States)
Marcus A. Maloof, Georgetown Univ. (United States)
Robert F. Wagner, FDA Ctr. for Devices and Radiological Health (United States)

Published in SPIE Proceedings Vol. 4684:
Medical Imaging 2002: Image Processing
Milan Sonka; J. Michael Fitzpatrick, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?