Share Email Print
cover

Journal of Medical Imaging • Open Access

Influence of study design in receiver operating characteristics studies: sequential versus independent reading
Author(s): Steven Schalekamp; Bram van Ginneken; Cornelia M. Schaefer-Prokop; Nico Karssemeijer

Paper Abstract

Observer studies to assess new image processing devices or computer-aided diagnosis techniques are often performed, but little is known about the effect of the study design on observer performance results. We investigated the effect of the sequential and independent reading design on observer study results with respect to reader performance and their statistical power. For this we performed an observer study for the detection of lung nodules with bone-suppressed images (BSIs) compared with original chest radiographs. In a fully crossed observer study, eight observers assessed a series of 300 radiographs four times, including one assessment of the original radiograph with sequential BSI and two independent reading sessions with BSI. Observer performance was compared using multireader multicase receiver operating characteristics. No significant difference between the effect of BSI in the sequential and the independent reading sessions could be found (p=0.09 ; p=0.46 ). Compared with the original radiographs, increased performance with BSI was significant in the sequential and one of the independent reading sessions (p<0.0001 ; p=0.0007 ), and nonsignificant in the other independent reading session (p=0.10 ). A strong increase of uncorrelated variance components was found in the independent reading sessions, masking the ability to demonstrate differences in observer performance across modalities. Therefore, the sequential reading design is the preferred design because it is less burdensome and has more statistical power.

Paper Details

Date Published: 23 April 2014
PDF: 8 pages
J. Med. Img. 1(1) 015501 doi: 10.1117/1.JMI.1.1.015501
Published in: Journal of Medical Imaging Volume 1, Issue 1
Show Author Affiliations
Steven Schalekamp, Radboud Univ. Nijmegen Medical Ctr. (Netherlands)
Bram van Ginneken, Radboud Univ. Nijmegen Medical Ctr. (Netherlands)
Cornelia M. Schaefer-Prokop, Radboud Univ. Nijmegen Medical Ctr. (Netherlands)
Meander Medical Ctr. (The Netherlands)
Nico Karssemeijer, Radboud Univ. Nijmegen Medical Ctr. (Netherlands)


© SPIE. Terms of Use
Back to Top