ISCA Archive AQS 2003
ISCA Archive AQS 2003

Paired-comparison rating of sound quality using map parameter estimation for data analysis

Martin Dahlquist, Arne Leijon

A method for paired-comparison evaluation of subjective sound quality has been implemented and tested in a listening experiment. The test procedure is intended for the rating of e.g. hearing instruments or any other audio-coding or audio presentation systems. We now present results from an experiment with 31 persons evaluating the fidelity of MP3-compressed versions of music material. Bit-rates of 192, 128, 96, and 64 kbit/s were used in comparison with the original CD version. Results are presented for three types of music material that were evaluated by 6-10 persons each.

Two methods of data analysis are applied and compared; one non-parametric method yielding results only on a rankorder scale, and one parametric method which places the tested systems on an interval scale defined by a Gaussian linear model.

Results show that the paired-comparison procedure could clearly quantify the quality degradation at 64 kbit/s for the three of the music materials presented here, and at 96 kbit/s for two of them. The degradation at 192 and 128 kbit/s is very subtle, and many test subjects could not distinguish these versions from the original. The median difference, across persons, between the original and the 192 kbit/s version for the three music materials were only 0.18, 0,04, and 0,001 units on the Gaussian linear scale. These differences mean that an average listener would prefer the original with a probability of about 55% or less, i.e. just slightly more than a random choice.


Cite as: Dahlquist, M., Leijon, A. (2003) Paired-comparison rating of sound quality using map parameter estimation for data analysis. Proc. First ISCA Workshop on Auditory Quality of Systems, 79-84

@inproceedings{dahlquist03_aqs,
  author={Martin Dahlquist and Arne Leijon},
  title={{Paired-comparison rating of sound quality using map parameter estimation for data analysis}},
  year=2003,
  booktitle={Proc. First ISCA Workshop on Auditory Quality of Systems},
  pages={79--84}
}