ESCA Workshop on Audio-Visual Speech Processing (AVSP'97)
September 26-27, 1997
The speech information conveyed by the shape of the mouth differs according to the characteristic phonetic system of each language. Besides the phonetic characteristic, a peculiar feature found in Japanese speech is that there are many homophenouswords (words having different sound but the same mouth shape), because the Japanese vocabulary comprises a large number of homophones (words having different sound but the same mouth shape), compared with other languages. This results in a decrease of information obtainable through speechreading. In order to ascertain this feature quantitatively, firstly, the mouth shapes of vowels and consonants of Japanese speech are described symbolically. Detailed symbols are assigned to as many kinds of mouth shapes as are readable in a clear utterance. Using these symbols, the changes in mouth shapes in Japanese speech are described based on coarticulation rules and a combination of preceding and following vowels. Then, the ratios of homophenous words and homophones found in a basic vocabulary are estimated by referring to statistical data of the syllabic structure of Japanese words. By assembling these ratios, the nature of the information conveyed through speechreading is analyzed, and the negative effect of homophones on the speechreading of Japanese is discussed.
Bibliographic reference. Hiki, Shizuo / Fukuda, Yumiko (1997): "Negative effect of homophones on speechreading in Japanese", In AVSP-1997, 9-12.