Share Email Print

Proceedings Paper

Interpreting the ROC curve from the radiologist’s point of view: the diagnostician operating choice (DOC) curve
Author(s): Craig A. Beam; Emily F. Conant M.D.; Elizabeth A. Krupinski; Harold L. Kundel; Edward A. Sickles
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

We introduce an interesting interpretation of the ROC Curve that, subsequently, opens a new research paradigm. We define the "Diagnostician Operating Choice" (DOC) Curve to be the set of all (True Positive Probability/True Negative Probability) or ("skill in diseased population"/"skill in non-diseased population" when considered from the diagnostician's perspective) options made available to a particular radiologist when interpreting a particular diagnostic technology. The DOC Curve is, thus, the choice set presented to the diagnostician by their interaction with the technology. This new paradigm calls for tools that can measure the particular choice set of any particular individual radiologist interpreting a particular technology when applied in a particular clinical setting. Fundamental requirements for this paradigm are for the DOC Curve to be unique to individuals and constant across similar experimental conditions. To investigate constancy, we analyzed data from a reading study of 10 radiologists. Each radiologist interpreted the same set of 148 screening mammograms twice using a modified version of BI-RADS. ROC Curves for each radiologist were computed and compared between the two reading occasions with the CORROC2 program. None of the areas were statistically significantly different (p<0.05), providing confirmation (but not proof) of constancy across the two reading conditions. The DOC Curve paradigm suggests new areas of research focusing on the behavior in individuals interacting with technology. A clear need is for more efficient estimation of individual DOC Curves based on limited case sets. Paradoxically, the answer to this last problem might lie in using large population-based ("MRMC") studies to develop highly efficient and externally validated standardized testing tools for assessment of the individual.

Paper Details

Date Published: 6 April 2005
PDF: 7 pages
Proc. SPIE 5749, Medical Imaging 2005: Image Perception, Observer Performance, and Technology Assessment, (6 April 2005);
Show Author Affiliations
Craig A. Beam, Univ. of South Florida (United States)
Emily F. Conant M.D., Univ. of Pennsylvania (United States)
Elizabeth A. Krupinski, Univ. of Arizona (United States)
Harold L. Kundel, Univ. of Pennsylvania (United States)
Edward A. Sickles, Univ. of California/San Francisco (United States)

Published in SPIE Proceedings Vol. 5749:
Medical Imaging 2005: Image Perception, Observer Performance, and Technology Assessment
Miguel P. Eckstein; Yulei Jiang, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?