Optimal SDRAM Buffer Allocator for Efficient Reuse of Layer IO in CNNs Inference Framework

Deep Learning based applications are becoming increasingly ubiquitous. The new generation smart phones are adapting lot of applications built on deep learning technology. However, adapting complex Deep Neural Network (DNN) applications on embedded processors is a huge challenge not only due to huge computational requirement, but also due to the massive SDRAM memory requirements of network layer IO buffers. Hence, an efficient reuse of layer IO buffers is required. However, it is challenging to reuse layer IO buffers because of complex network topology and large number of layers in the network. In this paper, we present an optimal SDRAM buffer allocator to minimize the overall SDRAM memory requirement of layer IO buffers, which works for any complex networks. The proposed SDRAM buffer allocator is integrated with ICNN (Inference only Convolutional Neural Networks) framework which is an extension of Caffe framework. Our framework with optimal SDRAM buffer allocator is experimented with popular AlexNet, GoogLeNet, ResNet-50 and Inception-ResNet-v2 CNNs. The results show 2× to 30× reduction in SDRAM footprint when compared with Caffe/TensorFlow frameworks and 26% to 47% reduction when compared with MXNet framework.