5th International Conference on Spoken Language Processing

Sydney, Australia
November 30 - December 4, 1998

Computer-Based Second Language Production Training By Using Spectrographic Representation And HMM-Based Speech Recognition Scores

Reiko Akahane-Yamada, Erik McDermott, Takahiro Adachi, Hideki Kawahara, John S. Pruitt

ATR Human Information Processing Research Laboratories, Japan

How can we provide feedback to second language (L2) learners about the goodness of their productions in an automatic way? In this paper, we introduce our attempts to provide effective feedback when we train native speakers of Japanese to produce English /r/ and /l/. First, we adopted spectrographic representation overlayed with formant frequencies as feedback. Second, we investigated the correlation between human judgments of L2 production quality and acoustic scores produced by an HMM-based speech recognition system. We also adopted the HMM-based scores as feedback in the production training. Evaluation of the pre- and post-training productions by human judges showed that production abilities of the trainees improved in both training groups, suggesting that both spectrographic representation and HMM-based scores were useful and meaningful as feedback. These results are discussed in the context of optimizing L2 speech training.

Full Paper

Bibliographic reference.  Akahane-Yamada, Reiko / McDermott, Erik / Adachi, Takahiro / Kawahara, Hideki / Pruitt, John S. (1998): "Computer-based second language production training by using spectrographic representation and HMM-based speech recognition scores", In ICSLP-1998, paper 0429.