Based on behavioral evidence, audiovisual speech perception is generally thought to proceed linearly from initial unimodal perceptual processing to integration of the unimodally processed information. We investigated unimodal versus audiovisual speech processing using electrical event-related potentials (ERPs) obtained from twelve adults. Nonsense syllable stimuli were presented in an oddball paradigm to evoke the mismatch negativity (MMN). Conditions were (1) audiovisual incongruent stimuli (visual /ga/ + auditory /ba/) versus congruent audiovisual stimuli (visual /ba/ + auditory /ba/), (2) visual-only stimuli from the audiovisual condition (/ga/ vs. /ba/), and (3) auditory-only stimuli (/ba/ vs. /da/). A visual- alone MMN was obtained on occipital and temporo-parietal electrodes, and the classical auditory MMN was obtained at the vertex electrode, Cz. Under audiovisual conditions, the negativity recorded at the occipital electrode locations was reduced in amplitude and latency compared to that recorded in the visual-only condition. Also, under the audiovisual condition, the vertex electrode showed a smaller negativity with increased latency relative to the auditory MMN. The neurophysiological evidence did not support a simple bottom-up linear flow from unimodal processing to audiovisual integration.
Cite as: Bernstein, L.E., Ponton, C.W., Auer Jr., E.T. (2001) Electrophysiology of unimodal and audiovisual speech perception. Proc. Auditory-Visual Speech Processing, 104-109
@inproceedings{bernstein01b_avsp,
author={Lynne E. Bernstein and Curtis W. Ponton and Edward T. {Auer Jr.}},
title={{Electrophysiology of unimodal and audiovisual speech perception}},
year=2001,
booktitle={Proc. Auditory-Visual Speech Processing},
pages={104--109}
}