Share Email Print
cover

Proceedings Paper

Pooling MRMC forced-choice data
Author(s): Brandon D. Gallas; Gene A. Pennello
Format Member Price Non-Member Price
PDF $14.40 $18.00

Paper Abstract

There are at least two sources of variance when estimating the performance of an imaging device: the doctors (readers) and the patients (cases). These sources of variability generate variances and covariances in the observer study data that can be addressed with multi-reader, multi-case (MRMC) variance analysis. Frequently, a fully-crossed study design is used to collect the data; every reader reads every case. For imaging devices used during in vivo procedures, however, a fully-crossed design is infeasible. Instead, each patient is diagnosed by only one doctor, a doctor-patient study design. Here we investigate percent correct (PC) under this doctor-patient study design. From a probabilistic foundation, we present the bias and variance of two statistics: pooled PC and reader-averaged PC. We also present variance estimates of these statistics and compare them to naive estimates. Finally, we run simulations to assess the statistics and the variance estimates. The two PC statistics have the same means but different variances. The variances depend on how patients are distributed among the readers and the amount of reader variability. Regarding the variance estimates, the MRMC estimates are unbiased, whereas the naive estimates bracket the true variance and can be extremely biased.

Paper Details

Date Published: 30 March 2007
PDF: 12 pages
Proc. SPIE 6515, Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment, 651506 (30 March 2007); doi: 10.1117/12.709628
Show Author Affiliations
Brandon D. Gallas, NIBIB/CDRH Lab. for the Assessment of Medical Imaging Systems (United States)
Gene A. Pennello, NIBIB/CDRH Lab. for the Assessment of Medical Imaging Systems (United States)


Published in SPIE Proceedings Vol. 6515:
Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment
Yulei Jiang; Berkman Sahiner, Editor(s)

© SPIE. Terms of Use
Back to Top