ODYSSEY 2004 - The Speaker and Language Recognition Workshop

May 31 - June 3, 2004
Toledo, Spain

The Expected Performance Curve: A New Assessment Measure for Person Authentication

Samy Bengio, Johnny Mariéthoz

IDIAP, Martigny, Switzerland

ROC and DET curves are often used in the field of person authentication to assess the quality of a model or even to compare several models. We argue in this paper that this measure can be misleading as it compares performance measures that cannot be reached simultaneously by all systems. We propose instead new curves, called Expected Performance Curves (EPC). These curves enable the comparison between several systems according to a criterion, decided by the application, which is used to set thresholds according to a separate validation set. A free sofware is available to compute these curves. A real case study is used throughout the paper to illustrate it. Finally, note that while this study was done on an authentication problem, it also applies to most 2-class classification tasks.

Full Paper

Bibliographic reference.  Bengio, Samy / Mariéthoz, Johnny (2004): "The expected performance curve: a new assessment measure for person authentication", In ODYS-2004, 279-284.