An intelligent bandwidth manager for CNN applications on embedded devices

Adapting complex Convolution Neural Network (CNN) applications on embedded processors is a challenge due to the massive memory bandwidth and computational requirements. In particular, the CNN memory bandwidth requirement poses a huge challenge for the processors with Scratch Pad Memory (SPM), usually of limited size. In this paper, we present an Intelligent Bandwidth Manager (IBWM) to efficiently handle the CNN bandwidth for SPM based processors. The proposed IBWM is a two fold approach which includes Intelligent SPM Manager (ISM) to optimize the number of accesses to SDRAM by analysing the data patterns, and Feature Map Compression (FMC) to further reduce the bandwidth by exploiting the feature map data sparsity. The IBWM is independent of any processor architecture and can be adopted in any processor with SPM. The proposed IBWM is experimented with ResNet-50 [1] and AlexNet [2] networks on a Samsung Reconfigurable Processor (SRP) [3] for various SPM sizes. The SDRAM bandwidth results show, 2x improvement compared to MIT Eyeriss [4] for AlexNet, and 4x-8x improvement compared to primitive bandwidth management techniques for AlexNet and ResNet-50. The proposed method achieves the bandwidth closer to the minimum possible bandwidth.