Share Email Print

Proceedings Paper

Optimal use of rating scales in ROC analysis
Author(s): R. M. Dawood; Andrew Todd-Pokropek; J. O.M.C. Craig; J. H. Highman; A. W. Porter
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Observers participating in ROC studies are usually required to estimate the confidence with which each observation is made. With a discrete scale, the rating, or score, normally falls into one of 5categories, ranging from 'definitely normal' to 'definitely abnormal'. However, a major problem in data analysis from ROC studies has been found to be caused by observers who have not used the rating scale in a uniform manner, and have made many responses corresponding to the two extreme categories with few responses falling in the middle. The use of a continuous rating scale, with a point selected using a mouse, has assisted in analysis, but only to a limited extent. It has therefore been suggested elsewhere that it is desirable to force observers to select intermediate points. The effect of such an approach on ROC curves was studied by asking a group of observers to re-score a set of difficult clinical images, after training and with continuous feedback on their compliance. Although the resulting fall in the ROC curves was not statistically significant, it is considered unwise to force observers to report in what to them appears to be an unnatural manner.

Paper Details

Date Published: 1 August 1990
PDF: 5 pages
Proc. SPIE 1234, Medical Imaging IV: PACS Systems Design and Evaluation, (1 August 1990); doi: 10.1117/12.19029
Show Author Affiliations
R. M. Dawood, St. Mary's Hospital (United Kingdom)
Andrew Todd-Pokropek, Univ. College London (United Kingdom)
J. O.M.C. Craig, St. Mary's Hospital (United Kingdom)
J. H. Highman, St. Mary's Hospital (United Kingdom)
A. W. Porter, St. Mary's Hospital (United Kingdom)

Published in SPIE Proceedings Vol. 1234:
Medical Imaging IV: PACS Systems Design and Evaluation
Samuel J. Dwyer; R. Gilbert Jost, Editor(s)

© SPIE. Terms of Use
Back to Top