ODYSSEY 2004 - The Speaker and Language Recognition Workshop May 31 - June 3, 2004 Toledo, Spain

Application-Independent Evaluation of Speaker Detection

Niko Brümmer

Spescom DataVoice, Stellenbosch, South Africa

We present a Bayesian analysis of the evaluation of speaker detection performance. We use expectation of utility to confirm that likelihood-ratio is both an optimum and application-independent form of output for speaker detection systems. We point out that the problem of likelihood-ratio calculation is equivalent to the problem of optimization of decision thresholds. It is shown that the decision cost that is used in the existing NIST evaluations effectively forms a utility (a proper scoring rule) for the evaluation of the quality of likelihood-ratio presentation. As an alternative, a logarithmic utility (a strictly proper scoring rule) is proposed. Finally, an information-theoretic interpretation of the expected logarithmic utility is given. It is hoped that this analysis and the proposed evaluation method will promote the use of likelihood-ratio detector output rather than decision output.

Full Paper

Bibliographic reference.  Brümmer, Niko (2004): "Application-independent evaluation of speaker detection", In ODYS-2004, 33-40.