Share Email Print

Proceedings Paper

Effect of observer inattention in a detection task on ROC analysis
Author(s): Yulei Jiang; William Sacks; Charles E. Metz
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

ROC experiments generally assume observers to be fully attentive to the task of the experiment and, therefore, that the experiment measures observers' innate ability in performing the task. In a detection task, inattention can cause an observer to overlook signals or signal-like potential false positives in an image. In an ROC experiment that involves a detection task, inattention can either cause an observer to report a strong confidence for signal absent, or cause the observer to report on some other finding less salient than the finding that the observer fails to notice. The purpose of this study was to determine the effect of observer inattention on empirical ROC estimates and the appropriateness of the conventional binormal model for fitting such ROC curves. An experiment was designed in which observers were asked to detect a signal of a simple geometric shape and varying contrast in a background of Gaussian noise. The images sometimes also contained other objects of different geometric shapes in locations that did not overlap the signal. Observer inattention was simulated by blocking a quarter of each image from observer view. Results showed that observer inattention caused the empirical ROC curve to decrease, but curve fitting with the binormal model appeared to be equally accurate with and without observer inattention. However, a "proper" ROC model might be more appropriate than the binormal model because the binormal model invariably produced a "hook" near the upper-right corner of the ROC unit square that was not indicated by the empirical operating points. We conclude that observer inattention in a detection task can be evaluated as part of a conventional ROC experiment.

Paper Details

Date Published: 6 April 2005
PDF: 4 pages
Proc. SPIE 5749, Medical Imaging 2005: Image Perception, Observer Performance, and Technology Assessment, (6 April 2005); doi: 10.1117/12.597597
Show Author Affiliations
Yulei Jiang, The Univ. of Chicago (United States)
William Sacks, Practicing Radiologist (United States)
Charles E. Metz, The Univ. of Chicago (United States)

Published in SPIE Proceedings Vol. 5749:
Medical Imaging 2005: Image Perception, Observer Performance, and Technology Assessment
Miguel P. Eckstein; Yulei Jiang, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?