Share Email Print
cover

Proceedings Paper

Enriching text with images and colored light
Author(s): Dragan Sekulovski; Gijs Geleijnse; Bram Kater; Jan Korst; Steffen Pauws; Ramon Clout
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

We present an unsupervised method to enrich textual applications with relevant images and colors. The images are collected by querying large image repositories and subsequently the colors are computed using image processing. A prototype system based on this method is presented where the method is applied to song lyrics. In combination with a lyrics synchronization algorithm the system produces a rich multimedia experience. In order to identify terms within the text that may be associated with images and colors, we select noun phrases using a part of speech tagger. Large image repositories are queried with these terms. Per term representative colors are extracted using the collected images. Hereto, we either use a histogram-based or a mean shift-based algorithm. The representative color extraction uses the non-uniform distribution of the colors found in the large repositories. The images that are ranked best by the search engine are displayed on a screen, while the extracted representative colors are rendered on controllable lighting devices in the living room. We evaluate our method by comparing the computed colors to standard color representations of a set of English color terms. A second evaluation focuses on the distance in color between a queried term in English and its translation in a foreign language. Based on results from three sets of terms, a measure of suitability of a term for color extraction based on KL Divergence is proposed. Finally, we compare the performance of the algorithm using either the automatically indexed repository of Google Images and the manually annotated Flickr.com. Based on the results of these experiments, we conclude that using the presented method we can compute the relevant color for a term using a large image repository and image processing.

Paper Details

Date Published: 28 January 2008
PDF: 15 pages
Proc. SPIE 6820, Multimedia Content Access: Algorithms and Systems II, 682004 (28 January 2008); doi: 10.1117/12.766421
Show Author Affiliations
Dragan Sekulovski, Philips Research Europe (Netherlands)
Gijs Geleijnse, Philips Research Europe (Netherlands)
Bram Kater, Philips Research Europe (Netherlands)
Jan Korst, Philips Research Europe (Netherlands)
Steffen Pauws, Philips Research Europe (Netherlands)
Ramon Clout, Philips Research Europe (Netherlands)


Published in SPIE Proceedings Vol. 6820:
Multimedia Content Access: Algorithms and Systems II
Theo Gevers; Ramesh C. Jain; Simone Santini, Editor(s)

© SPIE. Terms of Use
Back to Top