ISCA International Workshop on Speech and Language Technology in Education (SLaTE 2009)

Wroxall Abbey Estate, Warwickshire, England
September 3-5, 2009

Virtual Chinese Tutor (VCT) - A Chinese Language Pronunciation Learning Software

Yow-Bang Wang (1,2), Hsin-Min Wang (1), Lin-Shan Lee (1,2)

(1) Institute of Information Science, Academia Sinica
(2) Department of Electrical Engineering, National Taiwan University, Taipei, Taiwan, R.O.C.

Virtual Chinese Tutor (VCT) is a successfully operating online Chinese pronunciation learning software, specifically designed for giving the students opportunities to practice both their listening and speaking skills as many time as they wish anytime and anywhere. It was developed by a joint effort between Academia Sinica, National Taiwan University and some industry partners. The first version of VCT has been completed and made available on-line at Virtual Chinese Tutor (VCT) is able to evaluate the pronunciation of each utterance produced by an individual learner from four different aspects: pronunciation (i.e. the Initial (initial consonant) and Final (vowel or diphthong part plus an optional nasal ending) of each individual syllable (or character)), pitch (i.e. the lexical tone or neutral tone of each individual syllable (or character)), timing (i.e. the duration distribution among different syllables (or characters) in the utterance), and emphasis (i.e. the energy distribution among different syllables (or characters) in the utterance), as well as an overall score for the entire utterance. For those phonemes with scores below a threshold, a 3-dimensional video will show on the screen to demonstrate the actions of the vocal tract shape, including the relative positions among the lip, tongue and other articulators. After a learning session is finished, the diagnostic statistics for the learner is automatically summarized, indicating the important directions for improvements. This software platform can be used with any course content as long as the text and voice files for the course content are given. For each utterance produced by the learner, forced alignment with the corresponding utterance produced by the instructor is first performed, and the pronunciation, pitch, timing and emphasis scores for each syllable are then evaluated by a set of acoustic models, tone models, and prosodic models. The overall score is then the weighted sum of all those scores. The scoring algorithm was trained with the scores given by real professional Chinese teachers, over a corpus produced by a group of real learners whose mother tongues are not Chinese. Both the above training corpus and course content currently used in this software were contributed by the International Chinese Language Program of National Taiwan University. The details of the technologies will be explained when demonstrating the system in the technical program.
   VCT is now also a part of the program offered by NTUtorMing (, which is an online teaching institution that was jointly established by National Taiwan University (NTU) and TutorABC, a company located in Taipei focusing on language education. Chinese Learners around the world can have real-time interaction with professional Chinese teachers through TutorABC’s online learning platform, and VCT offers after-class practice or homework for the students.

Bibliographic reference.  Wang, Yow-Bang / Wang, Hsin-Min / Lee, Lin-Shan (2009): "Virtual Chinese tutor (VCT) - a Chinese language pronunciation learning software", In SLaTE-2009 (abstract).