Share Email Print

Proceedings Paper

A subjective study to evaluate video quality assessment algorithms
Author(s): Kalpana Seshadrinathan; Rajiv Soundararajan; Alan C. Bovik; Lawrence K. Cormack
Format Member Price Non-Member Price
PDF $17.00 $21.00

Paper Abstract

Automatic methods to evaluate the perceptual quality of a digital video sequence have widespread applications wherever the end-user is a human. Several objective video quality assessment (VQA) algorithms exist, whose performance is typically evaluated using the results of a subjective study performed by the video quality experts group (VQEG) in 2000. There is a great need for a free, publicly available subjective study of video quality that embodies state-of-the-art in video processing technology and that is effective in challenging and benchmarking objective VQA algorithms. In this paper, we present a study and a resulting database, known as the LIVE Video Quality Database, where 150 distorted video sequences obtained from 10 different source video content were subjectively evaluated by 38 human observers. Our study includes videos that have been compressed by MPEG-2 and H.264, as well as videos obtained by simulated transmission of H.264 compressed streams through error prone IP and wireless networks. The subjective evaluation was performed using a single stimulus paradigm with hidden reference removal, where the observers were asked to provide their opinion of video quality on a continuous scale. We also present the performance of several freely available objective, full reference (FR) VQA algorithms on the LIVE Video Quality Database. The recent MOtion-based Video Integrity Evaluation (MOVIE) index emerges as the leading objective VQA algorithm in our study, while the performance of the Video Quality Metric (VQM) and the Multi-Scale Structural SIMilarity (MS-SSIM) index is noteworthy. The LIVE Video Quality Database is freely available for download1 and we hope that our study provides researchers with a valuable tool to benchmark and improve the performance of objective VQA algorithms.

Paper Details

Date Published: 17 February 2010
PDF: 10 pages
Proc. SPIE 7527, Human Vision and Electronic Imaging XV, 75270H (17 February 2010); doi: 10.1117/12.845382
Show Author Affiliations
Kalpana Seshadrinathan, Intel Corp. (United States)
Rajiv Soundararajan, The Univ. of Texas at Austin (United States)
Alan C. Bovik, The Univ. of Texas at Austin (United States)
Lawrence K. Cormack, The Univ. of Texas at Austin (United States)

Published in SPIE Proceedings Vol. 7527:
Human Vision and Electronic Imaging XV
Bernice E. Rogowitz; Thrasyvoulos N. Pappas, Editor(s)

© SPIE. Terms of Use
Back to Top
Sign in to read the full article
Create a free SPIE account to get access to
premium articles and original research
Forgot your username?