Share Email Print

Proceedings Paper

Simulation of multi-reader multi-case study data with realistic ROC performance characteristics
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Multi-reader multi-case (MRMC) studies are widely used in assessing medical imaging and computer-aided diagnosis devices to demonstrate the generalizability of diagnostic performance to both the population of patient cases and the population of physician readers. Simulation of MRMC study data plays an important role in validating MRMC data analysis methods or sizing a pivotal study based on pilot data. The popular Roe and Metz simulation model is a linear mixed-effect model that models a human reader's latent decision variable in assessing patient's likelihood of disease as a sum of fixed modality effect, random reader effect, random case effect, random interaction effects, and a random error. The fixed effect is represented by a mean parameter and each random effect is represented by the variance parameter of a zero-mean Gaussian distribution. The purpose of this paper is to develop a method to set these parameters such that the simulated data have realistic ROC performance characteristics (mean AUC, variance components of AUC, and inter-reader/inter-modality correlations). To this end, we derived quasi-closed-form expressions to express the mean AUC and its U-statistic variance components as functions of the simulation model parameters. We then developed a numerical algorithm to solve the simulation model parameters from the mean AUC and its U-statistic variance components. Since the mean AUC and its U-statistic variance components can be estimated from real-world reader study data, simulated data have similar performance characteristics with the real-world data. Simulation studies were conducted to verify our parameter transformation algorithm.

Paper Details

Date Published: 16 March 2020
PDF: 6 pages
Proc. SPIE 11316, Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment, 113160M (16 March 2020); doi: 10.1117/12.2550545
Show Author Affiliations
Xiaochen Zhu, George Mason Univ. (United States)
Weijie Chen, U.S. FDA/CDRH/OSEL (United States)

Published in SPIE Proceedings Vol. 11316:
Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment
Frank W. Samuelson; Sian Taylor-Phillips, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?