Share Email Print

Proceedings Paper

Finite-sample effects and resampling plans: applications to linear classifiers in computer-aided diagnosis
Author(s): Robert F. Wagner; Heang-Ping Chan; Berkman Sahiner; Nicholas Petrick; Joseph T. Mossoba
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

This work provides an application and extension of the analysis of the effect of finite-sample training and test sets on the bias and variance of the classical discriminants as given by Fukunaga. The extension includes new results for the area under the ROC curve, Az. An upper bound on Az is provided by the so-called resubstitution method in which the classifier is trained and tested on the same patients; a lower bound is provided by the hold-out method in which the patient pool is partitioned into trainers and testers. Both methods exhibit a bias in Az with a linear dependence on the inverse of the number of patients Nt used to train the classifier; this leads to the possibility of obtaining an unbiased estimate of the infinite-population performance by a simple regression procedure. We examine the uncertainties in the resulting estimates. Whereas the bias of classifier performance is determined by the finite size of the training sample, the variance is dominated by the finite size of the test sample. This variance is approximately given by the simple result for an equivalent binomial process. A number of applications to the linear classifier are presented in this paper. More general applications, including the quadratic classifier and some elementary neural-network classifiers, are presented in a companion paper.

Paper Details

Date Published: 25 April 1997
PDF: 11 pages
Proc. SPIE 3034, Medical Imaging 1997: Image Processing, (25 April 1997); doi: 10.1117/12.274133
Show Author Affiliations
Robert F. Wagner, Ctr. for Devices and Radiological Health/FDA (United States)
Heang-Ping Chan, Univ. of Michigan (United States)
Berkman Sahiner, Univ. of Michigan (United States)
Nicholas Petrick, Univ. of Michigan (United States)
Joseph T. Mossoba, Ctr. for Devices and Radiological Health/FDA (United States)

Published in SPIE Proceedings Vol. 3034:
Medical Imaging 1997: Image Processing
Kenneth M. Hanson, Editor(s)

© SPIE. Terms of Use
Back to Top