12th Annual Conference of the International Speech Communication Association

Florence, Italy
August 27-31. 2011

Automatic Call Quality Monitoring Using Cost-Sensitive Classification

Youngja Park

IBM T.J. Watson Research Center, USA

In this paper, we propose advanced text analytics and cost-sensitive classification-based approaches for call quality monitoring and show that automatic quality monitoring with ASR transcripts can be achieved with a high accuracy. Our system analyzes ASR transcripts and determines if a call is a good call or a bad call. The set of features were identified through analysis of a large number of human monitoring results, which aim to estimate agent's attitude and customer's sentiment during the call. To enhance the accuracy of feature extraction, we apply various techniques to improve the quality of transcribed calls, such as sentence boundary detection and disfluency removal. We further note that quality monitoring has skewed class distribution and unequal classification error costs, and thus apply cost sensitive classification algorithms. Validation on 386 customer calls confirms the benefits of our approach. A SVM-based method produces a classification accuracy of 83.16% and 67.66% in F1 Score for identifying bad calls, which is promising. This system can therefore be used to conduct initial monitoring of all the calls in a contact center and to select calls that require human monitoring.

Full Paper

Bibliographic reference.  Park, Youngja (2011): "Automatic call quality monitoring using cost-sensitive classification", In INTERSPEECH-2011, 3085-3088.