Share Email Print

Proceedings Paper

The impact of using a JAFROC or ROC approach on the conclusions of a typical observer performance study
Author(s): Mohammad Rawashdeh; Warwick Lee; Mariusz Pietrzyk; Roger Bourne; Elaine Ryan; Warren Reed; Patrick C. Brennan
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Purpose: The current study aims to compare ROC with JAFROC methodologies to investigate how the choice of available analytical approaches in observer studies can impact upon study conclusions.
Methods and materials: A total of 129 readers independently reviewed 60 mammographic cases, 20 of which were biopsy proven cases (abnormal) and 40 were normal. Each case consisted of the four standard cranio-caudal (CC) and medio-lateral oblique (MLO) projections. Readers were asked to interpret and locate any presence of cancer, and levels of confidence were scored on a scale of 1-5. Radiology workstations supporting 5MP diagnostic monitors and with full image manipulation tools were used to display all images. JAFROC and ROC methodologies were used and figures of merit and Az values respectively were correlated against key reader characteristics such as experience, qualifications, breast reading practices and physical characteristics using Spearman techniques.
Results: Correlation analysis between reader characteristics and JAFROC analysis demonstrated that four key characteristics were linked to performance: years of qualification as a radiologist (p=0.05, r= 0.18), years reading mammograms (p=0.01, r=0.24), number of mammograms read per year (p=0.001, r=0.24), and hours reading mammogram per week (p=0.04, r= 0.19). The ROC method indicated that determinants of performance were confined to years reading mammograms (p=0.02, r = 0.2), and number of mammograms read per year (p=0.04, r=0.23).
Conclusion: This work demonstrates the practical impact on study conclusions when different methodologies are used. The location sensitivity approach employed and statistical power with JAFROC, would suggest that the findings from this approach should be prioritized.

Paper Details

Date Published: 28 March 2013
PDF: 6 pages
Proc. SPIE 8673, Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment, 86730B (28 March 2013); doi: 10.1117/12.2007360
Show Author Affiliations
Mohammad Rawashdeh, The Univ. of Sydney (Australia)
Warwick Lee, Cancer Institute NSW (Australia)
Mariusz Pietrzyk, The Univ. of Sydney (Australia)
Roger Bourne, The Univ. of Sydney (Australia)
Elaine Ryan, The Univ. of Sydney (Australia)
Warren Reed, The Univ. of Sydney (Australia)
Patrick C. Brennan, The Univ. of Sydney (Australia)

Published in SPIE Proceedings Vol. 8673:
Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment
Craig K. Abbey; Claudia R. Mello-Thoms, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?