Slide 1: CEVA Deep Neural Network
CEVA Deep Neural Network Liran Bar, Director of Product Marketing (Liran.Bar@ceva-dsp.com) July 2016 – Please contact us for the update material
Slide 2: Deep Learning Performance Improvements
Deep Learning Performance Improvements Human Performance Source: Nervana Systems CEVA Proprietary Information 2
Slide 3: The Neural Networks Challenge
The Neural Networks Challenge Various training frameworks: Caffe, TensorFlow, Torch etc. Various layers: Convolutional, Normalization, Pooling etc. Various network topologies Need to deal with network inside a network Inceptions Need to have optimized solution for variable sizes of ROI CEVA Propriet
Slide 4: Leading Deep Learning Frameworks
Leading Deep Learning Frameworks A well-known and widely used machine-vision library Implementation of fast convolutional nets to C and C++ Made with expression, speed, and modularity in mind Used by researchers, academy, and commercial companies Relatively new alternative to Caffe, supported and pr
Slide 5: Leading Neural Network Layers
Leading Neural Network Layers Convolutional Flexible embedded solution Normalization is required to cope with the Pooling (Average and Max) evolving and leading neural network layers Fully Connected Activation (ReLU, Parametric ReLU, TanH, Sigmoid) Deconvolution Concatenation Upsample Argmax Softmax
Slide 6: Deep Learning Topologies
Deep Learning Topologies Multiple Layers Per Linear Networks Multiple-Input-Multiple-Output Level Topology (a) (b) (c) AlexNet GoogLeNet GoogLeNet Networks VGG-19 SegNet VGG-16 ResNet VGG_S CEVA Proprietary Information 6
Slide 7: Neural Network Embedded Challenges
Neural Network Embedded Challenges Implementing a deep neural network in embedded systems is a challenging task! Very high bandwidth consuming and computing bottleneck Data transfer in/out the DDR – Input/output maps Convolution and Fully Connected data weights from DDR Processing multi ROI with the
Slide 8: GoogleNet Challenge
GoogleNet Challenge How do you run a network like GoogLeNet with minimum transaction through DRR? Inceptions CEVA Proprietary Information 8
Slide 9: What can be done?
What can be done? Reducing bandwidth In the convolution layer, each output is calculated by the same inputs Weights matrix are shared between output results in the same map (in order not to load the weights more than once) The input data can be reused to avoid useless transactions from DDR Convoluti
Slide 10: What can be done? – Cont.
What can be done? – Cont. Maximum multiply accumulate utilization Differentiate between large inputs to small input maps and the number of maps from each type 𝑙 𝐶 𝐻𝑊 𝑙 𝑙−1 Large size maps - 𝑋𝑐,𝑖,𝑗 = 𝑐=0 𝑗=0 𝑊𝑐,𝑚,𝑛 𝑋𝑐,𝑖+𝑚,𝑗+𝑛 𝑖=0 𝑙 Large amount of maps with small size (last…
Slide 11: What can be done? – Cont.
What can be done? – Cont. Overcome small internal memory size Try to preserve the principle of “All inputs must be in the internal memory” by tile division Divide all input maps to identical tile sizes Convolution n m CEVA Proprietary Information 11
Slide 12: What can be done? – Cont.
What can be done? – Cont. Use compression algorithms and prior knowledge to reduce bandwidth to and from the external memory Using algorithms like huffman coding Work in pipeline to save BW Identify when some of the calculation can be saved Share data between calculations Recognize when the focus sh
Slide 13: CEVA Deep Neural Network (CDNN2)
CEVA Deep Neural Network (CDNN2) 2nd gen SW framework support Caffe and TensorFlow Frameworks Various networks* All network topologies All the leading layers Variable ROI “Push-button” conversion from pre-trained networks to optimized real-time Accelerates machine learning deployment for embedded sy
Slide 14: CDNN2 Usage Flow
CDNN2 Usage Flow OEM / Partner (offline) CEVA (offline + real-time) CEVA Proprietary Information 14
Slide 15: Real-Time CDNN2 Application Flow
Real-Time CDNN2 Application Flow Input Image ROI “Horse” CEVA Proprietary Information 15
Slide 16: AlexNet - Network Performance
AlexNet - Network Performance Network specification Full forward classification case image measurement (single iteration) 24 layers, 224x224 network input size 11x11, 5x5 and 3x3 convolution filters Memory bandwidth Pre-trained network: 253Mbytes floating point Post CDNN2 (optimized for CEVA-XM4): 1
Slide 17: CEVA Network Generator
CEVA Network Generator Image Database Caffe Pre-Trained Network (For Training) TensorFlow (Deploy.caffemodel) Network Structure CEVA (Deploy.prototxt) CEVA-XM4 Network Optimized Generator Network Normalization Image (Deploy.binaryproto) Network Labeling (Deploy.labels) CEVA Proprietary Information 1
Slide 18: Real-Time Network Generator Demo
Real-Time Network Generator Demo Live CDNN2 demo: https://www.youtube.com/watch?v=SXINFryLM3Q&feature=youtu.be Downloading Age classification Passing it via CEVA Network Generator and Neural Network from the internet running it on the XM4 FPGA under 10 min ! CEVA Proprietary Information 18
Slide 19: Final Comments: HW + SW
Final Comments: HW + SW CEVA is at the forefront of development of Neural Network embedded platforms Normally, the HW platform is meaningless if not supported by the corresponding SW framework… New frameworks CDNN3 Other neural networks SW Framework TensorFlow Optimized for Next Gen CDNN2 Entire Caf
Slide 20: Thank You
Thank You