10th Annual Conference of the International Speech Communication Association

Brighton, United Kingdom
September 6-10, 2009

Automatic vs. Human Question Answering Over Multimedia Meeting Recordings

Quoc Anh Le (1), Andrei Popescu-Belis (2)

(1) University of Namur, Belgium
(2) IDIAP Research Institute, Switzerland

Information access in meeting recordings can be assisted by meeting browsers, or can be fully automated following a questionanswering (QA) approach. An information access task is defined, aiming at discriminating true vs. false parallel statements about facts in meetings. An automatic QA algorithm is applied to this task, using passage retrieval over a meeting transcript. The algorithm scores 59% accuracy for passage retrieval, while random guessing is below 1%, but only scores 60% on combined retrieval and question discrimination, for which humans reach 70%80% and the baseline is 50%. The algorithm clearly outperforms humans for speed, at less than 1 second per question, vs. 1.52 minutes per question for humans. The degradation on ASR compared to manual transcripts still yields lower but acceptable scores, especially for passage identification. Automatic QA thus appears to be a promising enhancement to meeting browsers used by humans, as an assistant for relevant passage identification.

Full Paper

Bibliographic reference.  Le, Quoc Anh / Popescu-Belis, Andrei (2009): "Automatic vs. human question answering over multimedia meeting recordings", In INTERSPEECH-2009, 624-627.