Share Email Print

Proceedings Paper

New scientific accuracy measure for performance evaluation of human-computer diagnostic systems
Author(s): Samuel C. Lee; Elisa T. Lee; Yiming Wang
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

This paper first presents a new scientific accuracy measure (denoted by G) for assessing/evaluating the performance of computer medical diagnostic (CMD) systems by incorporating the true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) of human and computer's diagnoses with respect to each other. Based on G, a formula for computing a multi-parameter sensitivity vector S(G), with the assumption that the system parameter percentage variations are small, is then proposed. For a given set of parameter percentage errors, from the expression of S(G), we can compute the error bound of G and assess the reliability of the system with human and/or computer errors being taken into consideration. It has been demonstrated that the new measure G is capable of providing consistent performance evaluation of a CMD system in general. Based on the value of G, a CMD system can be classified as having 'good', 'fair', or 'poor' performance. Even though the proposed basic accuracy measure and its sensitivity study are derived based on the diagnosis using two diagnostic categories (positive and negative) compared by two observers (a human expert and a computer system), however, its methodology can be extended to CMD systems with multiple diagnostic categories and observers. The formulas for measuring the performance of such systems are discussed and present.

Paper Details

Date Published: 25 September 2001
PDF: 12 pages
Proc. SPIE 4553, Visualization and Optimization Techniques, (25 September 2001); doi: 10.1117/12.441586
Show Author Affiliations
Samuel C. Lee, Univ. of Oklahoma (United States)
Elisa T. Lee, Univ. of Oklahoma Health Sciences Ctr. (United States)
Yiming Wang, Univ. of Oklahoma Health Sciences Ctr. (United States)

Published in SPIE Proceedings Vol. 4553:
Visualization and Optimization Techniques

© SPIE. Terms of Use
Back to Top