Share Email Print

Journal of Medical Imaging • Open Access

Generalized Roe and Metz receiver operating characteristic model: analytic link between simulated decision scores and empirical AUC variances and covariances

Paper Abstract

Modeling and simulation are often used to understand and investigate random quantities and estimators. In 1997, Roe and Metz introduced a simulation model to validate analysis methods for the popular endpoint in reader studies to evaluate medical imaging devices, the reader-averaged area under the receiver operating characteristic (ROC) curve. Here, we generalize the notation of the model to allow more flexibility in recognition that variances of ROC ratings depend on modality and truth state. We also derive and validate equations for computing population variances and covariances for reader-averaged empirical AUC estimates under the generalized model. The equations are one-dimensional integrals that can be calculated using standard numerical integration techniques. This work provides the theoretical foundation and validation for a Java application called iRoeMetz that can simulate multireader multicase ROC studies and numerically calculate the corresponding variances and covariances of the empirical AUC. The iRoeMetz application and source code can be found at the “iMRMC” project on the google code project hosting site. These results and the application can be used by investigators to investigate ROC endpoints, validate analysis methods, and plan future studies.

Paper Details

Date Published: 25 September 2014
PDF: 9 pages
J. Med. Imag. 1(3) 031006 doi: 10.1117/1.JMI.1.3.031006
Published in: Journal of Medical Imaging Volume 1, Issue 3
Show Author Affiliations
Brandon D. Gallas, U.S. Food and Drug Administration (United States)
Stephen L. Hillis, The Univ. of Iowa (United States)
CADRE (United States)

© SPIE. Terms of Use
Back to Top