Auditory-Visual Speech Processing 2007 (AVSP2007)

Kasteel Groenendaal, Hilvarenbeek, The Netherlands
August 31 - September 3, 2007

Speechreading in Context: An ERP Study

Marco Calabresi (1,2), Sharon M. Thomas (1), Tim J. Folkard (1), Deborah A. Hall (1)

(1) MRC Institute of Hearing Research, Nottingham, UK
(2) Institute of Neuroscience, University of Nottingham, UK

Aim - To determine whether single-word speechreading is sensitive to prior phrase context. Background - We used a well-established Event Related Potential (ERP) paradigm to elicit the N400, a negative wave around 400 ms. The N400 reflects activation by lexical semantic features and is modulated by contextual integration. Methods - Seventeen adult native English speakers participated. They were shown written three-word contexts followed by a final word (cloze) presented as a silent video clip. Context and cloze formed typical phrases (60 trials, e.g. "No smoke without fire") and anomalous phrases (120 trials, e.g. "Two of a trade"). Subjects were instructed to identify the cloze at the end of each trial. Typicality (typical/anomalous) and accuracy were used to classify trials. ERPs were computed off-line by averaging epochs aligned to the onset of articulatory movements. Results and discussion - For all four conditions, reliable evoked activity emerged only after 500 ms post-onset. Topographical analysis revealed a symmetrical sustained negative field at the frontal sites and positive field at the posterior sites. Statistical comparisons showed that only the waveform for the typical-correct condition differed from the other three. The difference, however, was not compatible with an N400. This finding suggests that the speechreading process integrates lexical semantic features differently from spoken and written language perception. We speculate on those cognitive mechanisms which may be responsible for the observed effects.

Full Paper

Bibliographic reference.  Calabresi, Marco / Thomas, Sharon M. / Folkard, Tim J. / Hall, Deborah A. (2007): "Speechreading in context: an ERP study", In AVSP-2007, paper L1-2.