Share Email Print

Proceedings Paper

A weighted quadratic asymptotic analysis of cost functions used in classifier design with extensions to finite-size training sets
Author(s): Gerald J. Dobeck
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

An analysis of the impact of cost function on classifier design is presented. The well know asymptotic probabilistic approach, that invokes the law of large numbers, is extended by incorporating a piece-wise weighted quadratic approximation. This allows different cost functions to be compared and better quantifies the impact of the cost function on the resulting classifier design. In this paper we show how the choice of several well known cost functions are related to (1) Bayesian optimality, (2) classifier complexity, and (3) the ability to estimate decision boundaries. This work extends previous work that relates classifier design to approximations of the Bayesian posterior probability of class membership (e.g., "Any Reasonable Cost Function Can be Used for A Posteriori Probability Approximation" by M. Saerens, et al., IEEE Transactions on NN, September 2002). Several cost functions are analyzed in the paper including the Lp norm and the maximum mutual information (MMI) criterion. An interesting example that supports the theoretical analysis is presented. For the example the Lp norm (with p=1.1) was shown to successfully estimate the Bayesian optimal class decision boundary while the MMI and the L2 criteria did not. In addition, a finite-version of the theory is presented that bridges the gap between asymptotic theory and strictly finite-size training sets.

Paper Details

Date Published: 9 April 2007
PDF: 14 pages
Proc. SPIE 6576, Independent Component Analyses, Wavelets, Unsupervised Nano-Biomimetic Sensors, and Neural Networks V, 657611 (9 April 2007); doi: 10.1117/12.720614
Show Author Affiliations
Gerald J. Dobeck, Naval Surface Warfare Ctr. Panama City (United States)

Published in SPIE Proceedings Vol. 6576:
Independent Component Analyses, Wavelets, Unsupervised Nano-Biomimetic Sensors, and Neural Networks V
Harold H. Szu; Jack Agee, Editor(s)

© SPIE. Terms of Use
Back to Top