Share Email Print

Proceedings Paper

Video attention deviation estimation using inter-frame visual saliency map analysis
Author(s): Yunlong Feng; Gene Cheung; Patrick Le Callet; Yusheng Ji
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

A viewer's visual attention during video playback is the matching of his eye gaze movement to the changing video content over time. If the gaze movement matches the video content (e.g., follow a rolling soccer ball), then the viewer keeps his visual attention. If the gaze location moves from one video object to another, then the viewer shifts his visual attention. A video that causes a viewer to shift his attention often is a "busy" video. Determination of which video content is busy is an important practical problem; a busy video is difficult for encoder to deploy region of interest (ROI)-based bit allocation, and hard for content provider to insert additional overlays like advertisements, making the video even busier. One way to determine the busyness of video content is to conduct eye gaze experiments with a sizable group of test subjects, but this is time-consuming and costineffective. In this paper, we propose an alternative method to determine the busyness of video-formally called video attention deviation (VAD): analyze the spatial visual saliency maps of the video frames across time. We first derive transition probabilities of a Markov model for eye gaze using saliency maps of a number of consecutive frames. We then compute steady state probability of the saccade state in the model-our estimate of VAD. We demonstrate that the computed steady state probability for saccade using saliency map analysis matches that computed using actual gaze traces for a range of videos with different degrees of busyness. Further, our analysis can also be used to segment video into shorter clips of different degrees of busyness by computing the Kullback-Leibler divergence using consecutive motion compensated saliency maps.

Paper Details

Date Published: 15 February 2012
PDF: 8 pages
Proc. SPIE 8305, Visual Information Processing and Communication III, 83050H (15 February 2012); doi: 10.1117/12.907384
Show Author Affiliations
Yunlong Feng, The Graduate Univ. for Advanced Studies (Japan)
Gene Cheung, National Institute of Informatics (Japan)
Patrick Le Callet, Univ. de Nantes (France)
Yusheng Ji, National Institute of Informatics (Japan)

Published in SPIE Proceedings Vol. 8305:
Visual Information Processing and Communication III
Amir Said; Onur G. Guleryuz; Robert L. Stevenson, Editor(s)

© SPIE. Terms of Use
Back to Top