ISCA Archive Interspeech 2018
ISCA Archive Interspeech 2018

Investigating the Role of Familiar Face and Voice Cues in Speech Processing in Noise

Jeesun Kim, Sonya Karisma, Vincent Aubanel, Chris Davis

The speech of a familiar talker is better recognized in noise than an unfamiliar one, suggesting that listeners access talker-specific models to assist with degraded input. This study investigated whether a talker model could be accessed by presenting the face of a talker. In the experiment, participants were trained in recognizing three talkers’ faces and voices to ceiling-level. Participants were then given a speech in noise recognition task consisting of four talker conditions: familiar face then familiar voice; unfamiliar face then familiar voice, familiar face then unfamiliar voice; and unfamiliar face then unfamiliar voice. A talker familiarity effect was found, i.e., speech perception was more accurate in the familiar face and familiar voice condition than all other ones. A familiar voice did not produce a talker familiarity effect when paired with an unfamiliar face. The familiar face and unfamiliar voice condition had the poorest performance, indicating that pairing a familiar face and unfamiliar voice had a disruptive effect. The results suggest that listeners develop a talker model that includes details of both the voice and the face; and that accessing this model can in some circumstances be wholly determined by face cues.


doi: 10.21437/Interspeech.2018-1812

Cite as: Kim, J., Karisma, S., Aubanel, V., Davis, C. (2018) Investigating the Role of Familiar Face and Voice Cues in Speech Processing in Noise. Proc. Interspeech 2018, 2276-2279, doi: 10.21437/Interspeech.2018-1812

@inproceedings{kim18d_interspeech,
  author={Jeesun Kim and Sonya Karisma and Vincent Aubanel and Chris Davis},
  title={{Investigating the Role of Familiar Face and Voice Cues in Speech Processing in Noise}},
  year=2018,
  booktitle={Proc. Interspeech 2018},
  pages={2276--2279},
  doi={10.21437/Interspeech.2018-1812}
}