Share Email Print

Journal of Electronic Imaging

Performance measure of image and video quality assessment algorithms: subjective root-mean-square error
Format Member Price Non-Member Price
PDF $20.00 $25.00

Paper Abstract

Evaluating algorithms used to assess image and video quality requires performance measures. Traditional performance measures (e.g., Pearson’s linear correlation coefficient, Spearman’s rank-order correlation coefficient, and root mean square error) compare quality predictions of algorithms to subjective mean opinion scores (mean opinion score/differential mean opinion score). We propose a subjective root-mean-square error (SRMSE) performance measure for evaluating the accuracy of algorithms used to assess image and video quality. The SRMSE performance measure takes into account dispersion between observers. The other important property of the SRMSE performance measure is its measurement scale, which is calibrated to units of the number of average observers. The results of the SRMSE performance measure indicate the extent to which the algorithm can replace the subjective experiment (as the number of observers). Furthermore, we have presented the concept of target values, which define the performance level of the ideal algorithm. We have calculated the target values for all sample sets of the CID2013, CVD2014, and LIVE multiply distorted image quality databases.The target values and MATLAB implementation of the SRMSE performance measure are available on the project page of this study.

Paper Details

Date Published: 28 March 2016
PDF: 13 pages
J. Electron. Imag. 25(2) 023012 doi: 10.1117/1.JEI.25.2.023012
Published in: Journal of Electronic Imaging Volume 25, Issue 2
Show Author Affiliations
Mikko Nuutinen, Univ. of Helsinki (Finland)
Toni Virtanen, Univ. of Helsinki (Finland)
Jukka P. Häkkinen, Univ. of Helsinki (Finland)

© SPIE. Terms of Use
Back to Top