Share Email Print

Proceedings Paper

Agreement experiments: a method for quantitatively testing new medical image display approaches
Author(s): Richard Eugene Johnston; Bonnie C. Yankaskas; John R. Perry; Stephen M. Pizer; David J. Delany; L. A. Parker
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

New medical image display devices or processes are commonly evaluated by anecdotal reports or subjective evaluations which are informative and relatively easy to acquire but do not provide quantitative nieasures. On the other hand, experinients eniploying ROC analysis, yield quantitative measurements but are very laborious and demand pathological proof of outcome. We have designed and are employing a new approach, which we have termed "agreement experiments," to quantitatively test the equivalence of observer performance on two systems. This was specifically developed to test whether a radiologist using a new display technique, which has some clear advantages over the standard technique, will detect and interpret diagnostic signs as he would with the standard display technique. Agreement experiments use checklists and confidence ratings to measure how well two radiologists agree on the presence of diagnostic signs when both view images on the standard display. This yields a baseline measure of agreement. Agreement measurements are then obtained when the two radiologists view cases using the new display, or display method, compared to the standard technique. If the levels of agreement when one reads from the new and one reads from the standard display are not statistically different from the baseline measures of agreement, we conclude the two systems are equivalent in conveying diagnostic signs. We will report on an experiment using this test. The experiment compares the agreement of radiological findings for chest CT studies viewed on the conventional multiformat film/lightbox to agreement of radiological findings from chest CT images presented on a multiple screen video system. The study consists of 80 chest CT studies. The results were an 86% to 81% agreement between the two viewing modalities which fell within our criteria of showing agreement.

Paper Details

Date Published: 1 August 1990
PDF: 10 pages
Proc. SPIE 1234, Medical Imaging IV: PACS Systems Design and Evaluation, (1 August 1990); doi: 10.1117/12.18983
Show Author Affiliations
Richard Eugene Johnston, Univ. of North Carolina/Chapel Hill (United States)
Bonnie C. Yankaskas, Univ. of North Carolina/Chapel Hill (United States)
John R. Perry, Univ. of North Carolina/Chapel Hill (United States)
Stephen M. Pizer, Univ. of North Carolina/Chapel Hill (United States)
David J. Delany, Univ. of North Carolina/Chapel Hill (United States)
L. A. Parker, Univ. of North Carolina/Chapel Hill (United States)

Published in SPIE Proceedings Vol. 1234:
Medical Imaging IV: PACS Systems Design and Evaluation
Samuel J. Dwyer III; R. Gilbert Jost M.D., Editor(s)

© SPIE. Terms of Use
Back to Top