Share Email Print

Proceedings Paper

A comparison of ROC inferred from FROC and conventional ROC
Author(s): Mark F. McEntee; Stephen Littlefair; Mariusz W. Pietrzyk
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

This study aims to determine whether receiver operating characteristic (ROC) scores inferred from free-response receiver operating characteristic (FROC) were equivalent to conventional ROC scores for the same readers and cases. Forty-five examining radiologists of the American Board of Radiology independently reviewed 47 PA chest radiographs under at least two conditions. Thirty-seven cases had abnormal findings and 10 cases had normal findings. Half the readers were asked to first locate any visualized lung nodules, mark them and assign a level of confidence [the FROC mark-rating pair] and second give an overall to the entire image on the same scale [the ROC score]. The second half of readers gave the ROC rating first followed by the FROC mark-rating pairs. A normal image was represented with number 1 and malignant lesions with numbers 2-5. A jackknife free-response receiver operating characteristic (JAFROC), and inferred ROC (infROC) was calculated from the mark-rating pairs using JAFROC V4.1 software. ROC based on the overall rating of the image calculated using DBM MRMC software, which was also used to compare infROC and ROC AUCs treating the methods as modalities. Pearson’s correlations coefficient and linear regression were used to examine their relationship using SPSS, version 21.0; (SPSS, Chicago, IL). The results of this study showed no significant difference between the ROC and Inferred ROC AUCs (p≤0.25). While Pearson’s correlation coefficient was 0.7 (p≤0.01). Inter-reader correlation calculated from Obuchowski- Rockette covariance’s ranged from 0.43-0.86 while intra-reader agreement was greater than previously reported ranging from 0.68-0.82.

Paper Details

Date Published: 11 March 2014
PDF: 8 pages
Proc. SPIE 9037, Medical Imaging 2014: Image Perception, Observer Performance, and Technology Assessment, 903718 (11 March 2014); doi: 10.1117/12.2044341
Show Author Affiliations
Mark F. McEntee, The Univ. of Sydney (Australia)
Stephen Littlefair, The Univ. of Sydney (Australia)
Mariusz W. Pietrzyk, The Univ. of Sydney (Australia)

Published in SPIE Proceedings Vol. 9037:
Medical Imaging 2014: Image Perception, Observer Performance, and Technology Assessment
Claudia R. Mello-Thoms; Matthew A. Kupinski, Editor(s)

© SPIE. Terms of Use
Back to Top