Sixth International Conference on Spoken Language Processing (ICSLP 2000)

Beijing, China
October 16-20, 2000

Using Multiple Time Scales in the Framework of Multi-Stream Speech Recognition

Astrid Hagen, Hervé Bourlard

Dalle Molle Institute for Perceptual Artificial Intelligence (IDIAP), Martigny, Switzerland

In this paper, we present a new approach to incorporating multiple time scale information as independent streams in multi-stream processing. To illustrate the procedure, we take two different sets of multiple time scale features. In the first system, these are features extracted over variable sized windows of three and five times the original window size. In the second system, we take as separate input streams the commonly used difference features, i.e. the first and second order derivatives of the instantaneous features. In the same way, any other kinds of multiple time scale features could be employed. The approach is embedded in the recently introduced "full combination" approach to multi-stream processing in which, the phoneme probabilities from all possible combinations of streams are combined in a weighted sum. As an extension of this approach we have found that replacing the sum of probabilities by their product, in the same "all wise" context, can result in higher robustness. Capturing different information in each stream, and with the longer time scale features being more robust to noise, the multiple time scale multi-stream system gained a significant performance improvement in both clean speech and in real-environmental noise.


Full Paper

Bibliographic reference.  Hagen, Astrid / Bourlard, Hervé (2000): "Using multiple time scales in the framework of multi-stream speech recognition", In ICSLP-2000, vol.1, 349-352.