ISCA Archive Interspeech 2009
ISCA Archive Interspeech 2009

Classification-based strategies for combining multiple 5-w question answering systems

Sibel Yaman, Dilek Hakkani-Tür, Gokhan Tur, Ralph Grishman, Mary Harper, Kathleen R. McKeown, Adam Meyers, Kartavya Sharma

We describe and analyze inference strategies for combining outputs from multiple question answering systems each of which was developed independently. Specifically, we address the DARPA-funded GALE information distillation Year 3 task of finding answers to the 5-Wh questions (who, what, when, where, and why) for each given sentence. The approach we take revolves around determining the best system using discriminative learning. In particular, we train support vector machines with a set of novel features that encode systems’ capabilities of returning as many correct answers as possible. We analyze two combination strategies: one combines multiple systems at the granularity of sentences, and the other at the granularity of individual fields. Our experimental results indicate that the proposed features and combination strategies were able to improve the overall performance by 22% to 36% relative to a random selection, 16% to 35% relative to a majority voting scheme, and 15% to 23% relative to the best individual system.

doi: 10.21437/Interspeech.2009-691

Cite as: Yaman, S., Hakkani-Tür, D., Tur, G., Grishman, R., Harper, M., McKeown, K.R., Meyers, A., Sharma, K. (2009) Classification-based strategies for combining multiple 5-w question answering systems. Proc. Interspeech 2009, 2703-2706, doi: 10.21437/Interspeech.2009-691

  author={Sibel Yaman and Dilek Hakkani-Tür and Gokhan Tur and Ralph Grishman and Mary Harper and Kathleen R. McKeown and Adam Meyers and Kartavya Sharma},
  title={{Classification-based strategies for combining multiple 5-w question answering systems}},
  booktitle={Proc. Interspeech 2009},