Auditory-Visual Speech Processing
Speaking and fingerspelling words are dynamic, biological, visual activities that serve language communication. However, fingerspelling is a manual articulation of orthography, whereas the visible aspect of spoken language is a product of speech articulation. These two stimulus types provide a revealing contrast for examining the cortical substrate for language perception. Functional magnetic resonance imaging (fMRI) was used to investigate cortical activity due to viewing spoken (lipread) vs. fingerspelled words. In Experiment 1, young adults with prelingual-onset severe-to-profound hearing impairments were imaged. In Experiment 2, young adults with normal hearing and minimal previous experience with fingerspelling were imaged. In both participant groups, fingerspelling and lipreading activated regions of the superior temporal sulcus (STS). However, in the normal-hearing participants, fingerspelling activated fewer regions in the STS. In both participant groups, other activation was observed for fingerspelling, including several dorsolateral parietal areas. These results suggest that the perception of different forms of biological motion (spoken and fingerspelled) occurs in partially shared but partially distinct cortical networks, depending on linguistic significance/ experience of the perceiver.
Bibliographic reference. Auer, Edward T. Jr. / Bernstein, Lynne E. / Singh, Manbir (2001): "Comparing cortical activity during the perception of two forms of biological motion for language communication", In AVSP-2001, 40-44.