First ISCA ITRW on Auditory Quality of Systems

April 23-25, 2003
Akademie Mont-Cenis, Germany

Paired-Comparison Rating of Sound Quality Using MAP Parameter Estimation for Data Analysis

Martin Dahlquist, Arne Leijon

Department of Speech, Music and Hearing, KTH, Stockholm, Sweden

A method for paired-comparison evaluation of subjective sound quality has been implemented and tested in a listening experiment. The test procedure is intended for the rating of e.g. hearing instruments or any other audio-coding or audio presentation systems. We now present results from an experiment with 31 persons evaluating the fidelity of MP3-compressed versions of music material. Bit-rates of 192, 128, 96, and 64 kbit/s were used in comparison with the original CD version. Results are presented for three types of music material that were evaluated by 6-10 persons each.

Two methods of data analysis are applied and compared; one non-parametric method yielding results only on a rankorder scale, and one parametric method which places the tested systems on an interval scale defined by a Gaussian linear model.

Results show that the paired-comparison procedure could clearly quantify the quality degradation at 64 kbit/s for the three of the music materials presented here, and at 96 kbit/s for two of them. The degradation at 192 and 128 kbit/s is very subtle, and many test subjects could not distinguish these versions from the original. The median difference, across persons, between the original and the 192 kbit/s version for the three music materials were only 0.18, 0,04, and 0,001 units on the Gaussian linear scale. These differences mean that an average listener would prefer the original with a probability of about 55% or less, i.e. just slightly more than a random choice.

Full Paper

Bibliographic reference.  Dahlquist, Martin / Leijon, Arne (2003): "Paired-comparison rating of sound quality using MAP parameter estimation for data analysis", In AQS-2003, 79-84.