ISCA Archive Interspeech 2015
ISCA Archive Interspeech 2015

Efficient GPU implementation of convolutional neural networks for speech recognition

Ewout van den Berg, Daniel Brand, Rajesh Bordawekar, Leonid Rachevsky, Bhuvana Ramabhadran

Deep learning has enjoyed tremendous success in speech recognition in recent years. Despite their widespread use, training large and complex architectures remains very time consuming. A prime example of this are convolutional neural networks (CNNs), which have provided state-of-the-art results, but are also among the most computationally intensive networks to train. In this paper, we study four different methods for GPU acceleration of CNNs: a native implementation using cuBLAS, two implementations based on NVIDIA's recently released deep-learning cuDNN library, and an implementation based on cuFFT. We analyze the performance of each of these approaches on the forward operation, the gradient computation, and the backward propagation. The overall best performance is obtained using the custom native implementation, which was found to be up to 6.9 times faster than cuDNN. The paper concludes with results on the end-to-end training speed of our CNN network on an LVCSR task.

doi: 10.21437/Interspeech.2015-353

Cite as: Berg, E.v.d., Brand, D., Bordawekar, R., Rachevsky, L., Ramabhadran, B. (2015) Efficient GPU implementation of convolutional neural networks for speech recognition. Proc. Interspeech 2015, 1483-1487, doi: 10.21437/Interspeech.2015-353

  author={Ewout van den Berg and Daniel Brand and Rajesh Bordawekar and Leonid Rachevsky and Bhuvana Ramabhadran},
  title={{Efficient GPU implementation of convolutional neural networks for speech recognition}},
  booktitle={Proc. Interspeech 2015},