The hardware power-aware Keyword Spotting (KWS) implementation requires small memory footprint, low-complex computation, and high accuracy performances. In this article, three aspects are introduced to satisfy these three stringent requirements. Firstly, a lightweight Binary Residual Neural Network (B-ResNet) is proposed and applied to the small-footprint KWS. The parameters and calculations inside the net-work are greatly downscaled during the binary quantization. Secondly, during the forward propagation, distribution of the binary activation is optimized by our proposed learnable activation function with fix-valued shift initialization. Thirdly, our variable periodic window (PW) for the backward gradient correction (BGC) is also put forward to avoid gradient mismatch and vanishing problems during the back-propagation. These two improvements effectively increase the accuracy performance during the binarization. Our studies in this article are very helpful and promising for the future hardware KWS implementations.
Cite as: Wang, X., Cheng, S., Li, J., Qiao, S., Zhou, Y., Zhan, Y. (2022) Low-complex and Highly-performed Binary Residual Neural Network for Small-footprint Keyword Spotting. Proc. Interspeech 2022, 3233-3237, doi: 10.21437/Interspeech.2022-573
@inproceedings{wang22g_interspeech,
author={Xiao Wang and Song Cheng and Jun Li and Shushan Qiao and Yumei Zhou and Yi Zhan},
title={{Low-complex and Highly-performed Binary Residual Neural Network for Small-footprint Keyword Spotting}},
year=2022,
booktitle={Proc. Interspeech 2022},
pages={3233--3237},
doi={10.21437/Interspeech.2022-573}
}