ISCA Archive Interspeech 2023
ISCA Archive Interspeech 2023

HyperConformer: Multi-head HyperMixer for Efficient Speech Recognition

Florian Mai, Juan Zuluaga-Gomez, Titouan Parcollet, Petr Motlicek

State-of-the-art ASR systems have achieved promising results by modeling local and global interactions separately. While the former can be computed efficiently, global interactions are usually modeled via attention mechanisms, which are expensive for long input sequences. Here, we address this by extending HyperMixer, an efficient alternative to attention exhibiting linear complexity, to the Conformer architecture for speech recognition, leading to HyperConformer. In particular, multi-head HyperConformer achieves comparable or higher recognition performance while being more efficient than Conformer in terms of inference speed, memory, parameter count, and available training data. HyperConformer achieves a word error rate of 2.9% on LibriSpeech test-clean with less than 8M neural parameters and a peak memory during training of 5.7GB, hence trainable with accessible hardware. Inference speed is between 38% on mid-length speech and 56% on long speech faster than an equivalent Conformer.

doi: 10.21437/Interspeech.2023-1611

Cite as: Mai, F., Zuluaga-Gomez, J., Parcollet, T., Motlicek, P. (2023) HyperConformer: Multi-head HyperMixer for Efficient Speech Recognition. Proc. INTERSPEECH 2023, 2213-2217, doi: 10.21437/Interspeech.2023-1611

  author={Florian Mai and Juan Zuluaga-Gomez and Titouan Parcollet and Petr Motlicek},
  title={{HyperConformer: Multi-head HyperMixer for Efficient Speech Recognition}},
  booktitle={Proc. INTERSPEECH 2023},