Share Email Print

Journal of Electronic Imaging

Online subjective testing for consumer-photo quality evaluation
Author(s): Michele A. Saad; Patrick McKnight; Jacob Quartuccio; David Nicholas; Ramesh Jaladi; Philip Corriveau
Format Member Price Non-Member Price
PDF $20.00 $25.00
cover GOOD NEWS! Your organization subscribes to the SPIE Digital Library. You may be able to download this paper for free. Check Access

Paper Abstract

We take a look at crowdsourcing for subjective image quality evaluation using real image stimuli with nonsimulated distortions. Our aim is to scale the task of subjectively rating images while ensuring maximal data validity and accuracy. While previous work has begun to explore crowdsourcing for quality assessment, it has either used images that are not representative of popular consumer scenarios or used crowdsourcing to collect data without comparison to experiments in a controlled environment. Here, we address the challenges imposed by the highly variable online environment, using stimuli that are subtle and more complex than has traditionally been used in quality assessment experiments. In a series of experiments, we vary different design parameters and demonstrate how they impact the subjective responses obtained. Of the parameters examined are stimulus display mode, study length, stimulus habituation, and content homogeneity/heterogeneity. Our method was tested on a database that was rated in a laboratory test previously. Once our design parameters were chosen, we rated a database of consumer photographs and are making this data available to the research community.

Paper Details

Date Published: 15 July 2016
PDF: 13 pages
J. Electron. Imag. 25(4) 043009 doi: 10.1117/1.JEI.25.4.043009
Published in: Journal of Electronic Imaging Volume 25, Issue 4
Show Author Affiliations
Michele A. Saad, Intel Corp. (United States)
Patrick McKnight, George Mason Univ. (United States)
Jacob Quartuccio, George Mason Univ. (United States)
David Nicholas, Intel Corp. (United States)
Ramesh Jaladi, Intel Corp. (United States)
Philip Corriveau, Intel Corp. (United States)

© SPIE. Terms of Use
Back to Top