Share Email Print

Proceedings Paper

Improving video captioning for deaf and hearing-impaired people based on eye movement and attention overload
Author(s): C. Chapdelaine; V. Gouaillier; M. Beaulieu; L. Gagnon
Format Member Price Non-Member Price
PDF $14.40 $18.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

Deaf and hearing-impaired people capture information in video through visual content and captions. Those activities require different visual attention strategies and up to now, little is known on how caption readers balance these two visual attention demands. Understanding these strategies could suggest more efficient ways of producing captions. Eye tracking and attention overload detections are used to study these strategies. Eye tracking is monitored using a pupilcenter- corneal-reflection apparatus. Afterward, gaze fixation is analyzed for each region of interest such as caption area, high motion areas and faces location. This data is also used to identify the scanpaths. The collected data is used to establish specifications for caption adaptation approach based on the location of visual action and presence of character faces. This approach is implemented in a computer-assisted captioning software which uses a face detector and a motion detection algorithm based on the Lukas-Kanade optical flow algorithm. The different scanpaths obtained among the subjects provide us with alternatives for conflicting caption positioning. This implementation is now undergoing a user evaluation with hearing impaired participants to validate the efficiency of our approach.

Paper Details

Date Published: 12 February 2007
PDF: 11 pages
Proc. SPIE 6492, Human Vision and Electronic Imaging XII, 64921K (12 February 2007); doi: 10.1117/12.703344
Show Author Affiliations
C. Chapdelaine, Computer Research Institute of Montreal (Canada)
V. Gouaillier, Computer Research Institute of Montreal (Canada)
M. Beaulieu, Computer Research Institute of Montreal (Canada)
L. Gagnon, Computer Research Institute of Montreal (Canada)

Published in SPIE Proceedings Vol. 6492:
Human Vision and Electronic Imaging XII
Bernice E. Rogowitz; Thrasyvoulos N. Pappas; Scott J. Daly, Editor(s)

© SPIE. Terms of Use
Back to Top