Share Email Print

Proceedings Paper

Test set readings predict clinical performance to a limited extent: preliminary findings
Author(s): BaoLin P. Soh; Warwick M. Lee; Peter L. Kench; Warren M. Reed; Mark F. McEntee; Patrick C. Brennan
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Aim: To investigate the level of agreement between test sets and actual clinical reading
Background: The performance of screen readers in detecting breast cancer is being assessed in some countries by using mammographic test sets. However, previous studies have provided little evidence that performance assessed by test sets strongly correlate to performance in clinical reading.
Methods: Five clinicians from BreastScreen New South Wales participated in this study. Each clinician was asked to read 200 de-identified mammographic examinations gathered from their own case history within the BreastScreen NSW Digital Imaging Library. All test sets were designed with specific proportions of true positive, true negative, false positive and false negative examinations from the previous actual clinical reads of each reader. A prior mammogram examination for comparison (when available) was also provided for each case.
Results: Preliminary analyses have shown that there is a moderate level of agreement (Kappa 0.42−0.56, p < 0.001) between laboratory test sets and actual clinical reading. In addition, a mean increase of 38% in sensitivity in the laboratory test sets as compared to their actual clinical readings was demonstrated. Specificity is similar between the laboratory test sets and actual clinical reading.
Conclusion: This study demonstrated a moderate level of agreement between actual clinical reading and test set reading, which suggests that test sets have a role in reflecting clinical performance.

Paper Details

Date Published: 28 March 2013
PDF: 7 pages
Proc. SPIE 8673, Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment, 86730R (28 March 2013); doi: 10.1117/12.2000705
Show Author Affiliations
BaoLin P. Soh, The Univ. of Sydney (Australia)
Singapore General Hospital (Singapore)
Warwick M. Lee, Cancer Institute NSW (Australia)
Peter L. Kench, The Univ. of Sydney (Australia)
Warren M. Reed, The Univ. of Sydney (Australia)
Mark F. McEntee, The Univ. of Sydney (Australia)
Patrick C. Brennan, The Univ. of Sydney (Australia)

Published in SPIE Proceedings Vol. 8673:
Medical Imaging 2013: Image Perception, Observer Performance, and Technology Assessment
Craig K. Abbey; Claudia R. Mello-Thoms, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?