Share Email Print

Proceedings Paper

Frame rate of motion picture and its influence on speech perception
Author(s): Kaoru Nakazono
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

The preservation of QoS for multimedia traffic through a data network is a difficult problem. We focus our attention on video frame rate and study its influence on speech perception. When sound and picture are discrepant (e.g., acoustic `ba' combined with visual `ga'), subjects perceive a different sound (such as `da'). This phenomenon is known as the McGurk effect. In this paper, the influence of degraded video frame rate on speech perception was studied. It was shown that when frame rate decreases, correct hearing is improved for discrepant stimuli and is degraded for congruent (voice and picture are the same) stimuli. Furthermore, we studied the case where lip closure was always captured by the synchronization of sampling time and lip position. In this case, frame rate has little effect on mishearing for congruent stimuli. For discrepant stimuli, mishearing is decreased with degraded frame rate. These results indicate that stiff motion of lips resulting from low frame rate cannot give enough labial information for speech perception. In addition, the effect of delaying the picture to correct for low frame rate was studied. The results, however, were not as definitive as expected because of compound effects related to the synchronization of sound and picture.

Paper Details

Date Published: 25 March 1996
PDF: 10 pages
Proc. SPIE 2667, Multimedia Computing and Networking 1996, (25 March 1996); doi: 10.1117/12.235873
Show Author Affiliations
Kaoru Nakazono, NTT Software Labs. (Japan)

Published in SPIE Proceedings Vol. 2667:
Multimedia Computing and Networking 1996
Martin Freeman; Paul Jardetzky; Harrick M. Vin, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?