Share Email Print

Proceedings Paper

Multi-reader multi-case analysis of variance software for diagnostic performance comparison of imaging modalities
Author(s): Brian J. Smith; Stephen L. Hillis
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

A common study design for comparing the diagnostic performance of imaging modalities is to obtain modalityspecific ratings from multiple readers of multiple cases whose true statuses are known. Typically, there is overlap between the modalities, readers, and/or cases for which special analytical methods are needed to perform statistical comparisons. We describe our new R software package MRMCaov, which is designed for multi-reader multi-case comparisons of two or more imaging modalities. The software allows for the comparison of reader performance metrics, such as area under the receiver operating characteristic curve (ROC AUC), with analysis of variance methods originally proposed by Obuchowski and Rockette (1995) and later unified and improved by Hillis and colleagues (2005, 2007, 2008, 2018). MRMCaov is an open-source package with an integrated command-line interface for performing multi-reader multi-case statistical analysis, plotting, and presenting results. Features of the package include (1) ROC curves estimated parametrically or non-parametrically; (2) reader-specific ROC curves and performance metrics; (3) user-definable performance metrics; (4) modality-specific estimates of mean performance along with confidence intervals and p-values for statistical comparisons; (5) support for factorial, nested, or partially paired study designs; (6) inference for random readers and cases, random readers and fixed cases, or fixed readers and random cases; (7) DeLong, jackknife, or unbiased covariance estimation; and (8) compatibility with Microsoft Windows, Mac OS, and Linux.

Paper Details

Date Published: 16 March 2020
PDF: 8 pages
Proc. SPIE 11316, Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment, 113160K (16 March 2020); doi: 10.1117/12.2549075
Show Author Affiliations
Brian J. Smith, Univ. of Iowa (United States)
Stephen L. Hillis, Univ. of Iowa (United States)

Published in SPIE Proceedings Vol. 11316:
Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment
Frank W. Samuelson; Sian Taylor-Phillips, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?