Share Email Print

Proceedings Paper

Learning to annotate video databases
Format Member Price Non-Member Price
PDF $17.00 $21.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Model-based approach to video retrieval requires ground-truth data for training the models. This leads to the development of video annotation tools that allow users to annotate each shot in the video sequence as well as to identify and label scenes, events, and objects by applying the labels at the shot-level. The annotation tool considered here also allows the user to associate the object-labels with an individual region in a key-frame image. However, the abundance of video data and diversity of labels make annotation a difficult and overly expensive task. To combat this problem, we formulate the task of annotation in the framework of supervised training with partially labeled data by viewing it as an exercise in active learning. In this scenario, one first trains a classifier with a small set of labeled data, and subsequently updates the classifier by selecting the most informative, or most uncertain subset of the available data-set. Consequently, propagation of labels to yet unlabeled data is automatically achieved as well. The purpose of this paper is primarily twofold. The first is to describe a video annotation tool that has been developed for the purpose of annotating generic video sequences in the context of a recent video-TREC benchmarking exercise. The tool is semi-automatic in that it automatically propagates labels to similar shots, which requires the user to confirm or reject the propagated labels. The second purpose is to show how active learning strategy can be potentially implemented in this context to further improve the performance of the annotation tool. While many versions of active learning could be thought of, we specifically report results on experiments with support vector machine classifiers with polynomial kernels.

Paper Details

Date Published: 19 December 2001
PDF: 12 pages
Proc. SPIE 4676, Storage and Retrieval for Media Databases 2002, (19 December 2001); doi: 10.1117/12.451096
Show Author Affiliations
Milind Ramesh Naphade, IBM Thomas J. Watson Research Ctr. (United States)
Ching-Yung Lin, IBM Thomas J. Watson Research Ctr. (United States)
John R. Smith, IBM Thomas J. Watson Research Ctr. (United States)
Belle L. Tseng, IBM Thomas J. Watson Research Ctr. (United States)
Sankar Basu, IBM Thomas J. Watson Research Ctr. (United States)

Published in SPIE Proceedings Vol. 4676:
Storage and Retrieval for Media Databases 2002
Minerva M. Yeung; Chung-Sheng Li; Rainer W. Lienhart, Editor(s)

© SPIE. Terms of Use
Back to Top