Slide 1: Achieving AI @Scale on Mobile Devices
March 2018 @qualcomm_tech Achieving AI @Scale on Mobile Devices Qualcomm Technologies, Inc.
Slide 2: Mobile is the largest computing platform in the world
Mobile is the largest computing platform in the world ~ 7.8 Billion Cumulative smartphone unit shipments forecast between 2018–2022 Source: IDC Aug. ‘18 2
Slide 3: 30 Years of driving the evolution of wireless
30 Years of driving the evolution of wireless # 1 Fabless semiconductor company # 1 in 3G/4G LTE modem 842M MSM™ chipsets shipped FY’16 Source: Qualcomm Incorporated data. Currently, Qualcomm semiconductors are products of Qualcomm Technologies, Inc. or its subsidiaries. MSM is a product of Qualcomm
Slide 4: Qualcomm Technologies' success is based on technology leadership
Qualcomm Technologies’ success is based on Modem 3G (CDMA, GSM) Technology leadership Processors 4G/LTE CPU 5G GPU DSP Connectivity Bluetooth Smart Multimedia Bluetooth Mesh 802.11ac/ax Connectivity Computing Video Speech 802.11ad Display processing 802.11n Media processing 802.15.4 Camera processin
Slide 5: Chipset complexity is growing dramatically
Chipset complexity is growing dramatically More frequent and more complex design cycles Circa 2000 Today < 10 Million Transistors on a chip 3 > Billion Transistors on a chip 1 chipset per year ~10 chipsets per year Comparative scale Qualcomm Snapdragon is a product of Qualcomm Technologies, Inc. Sou
Slide 6: A mobile processor today—Snapdragon 835
A mobile processor today—Snapdragon 835 Highly integrated and complex SoC using 10nm process technology Qualcomm® Adreno™ Snapdragon X16 LTE Snapdragon X16 LTE modem Adreno 540 Graphics Processing Unit Visual Processing World’s first announced (GPU) 25% faster graphics rendering gigabit-class LTE mo
Slide 7: Mobile scale changes everything
Mobile computing Smart cities Mobile scale changes everything Automotive Smart homes Healthcare Industrial IoT Networking Wearables Extended reality Rapid Integrated and replacement cycles Superior scale optimized technologies 7
Slide 8: Intelligence is moving to the device
Intelligence is moving to the device Server/Cloud Devices Training Execution/Inference Execution/Inference Training (emerging) 8
Slide 9: On-device intelligence is paramount
On-device Privacy intelligence is Reliability paramount Low latency Process data closest to the source, complement the cloud Efficient use of network bandwidth 9
Slide 10: Mobile is becoming the pervasive AI platform
Mobile is becoming the pervasive AI platform 10
Slide 11: Qualcomm Technologies is accelerating on-device AI
Qualcomm Technologies is accelerating on-device AI Making efficient on-device machine learning possible for highly responsive, private, and intuitive user experiences 11
Slide 12: Qualcomm Artificial Intelligence Platform
Qualcomm Artificial A high-performance ® platform designed to Intelligence Platform support myriad intelligent- The platform for efficient on-device on-device-capabilities machine learning that utilize: • Qualcomm® Snapdragon™ Audio mobile platform’s heterogeneous intelligence compute capabilities w
Slide 13: Making on-device intelligence pervasive
Making on-device intelligence pervasive Focusing on high performance HW/SW and optimized network design Efficient Algorithmic Software hardware advancements tools Developing heterogeneous compute to Algorithmic research that benefits from Software accelerated run-time run demanding neural networks a
Slide 14: Algorithmic enhancements for space and runtime efficiency
Algorithmic enhancements for space and runtime efficiency Improve performance by addressing model complexity Neural network optimizations Required operations per image for embedded 40000 • Improved network architecture • Focus on memory and storage 30000 27x ◦ Reduce bit widths Reduction in complexi
Slide 15: Snapdragon Neural Processing SDK
Snapdragon Neural Processing SDK Software accelerated runtime for the execution of deep neural networks on device Efficient execution on Snapdragon • Takes advantage of Snapdragon heterogeneous computing capabilities Qualcomm ® Qualcomm ® Qualcomm ® • Runtime and libraries accelerate deep Kryo CPU ™
Slide 16: Robust software and tools simplify development
Robust software and tools simplify development Object classification Mobile Face detection NP SDK Auto GoogleNet/ Scene segmentation Inception SSD Natural language Home understanding Alexnet ResNet Speaker recognition Camera SqueezeNet Security/Authentication XR Resource management 16
Slide 17: Model to runtime workflow: Training and inference
Model to runtime workflow: Training and inference Training: Machine learning experts build and train their network to solve their particular problem Backward propagation No Model building and training Deep Training Good Model testing enough learning data Data 1 match? Yes frameworks (e.g., Caffe2, T
Slide 18: High-level software architecture for the Snapdragon NPE
High-level software architecture for the Snapdragon NPE Application code User developed NPE API Future Model loader NPE library Application Profiling logging User defined layers Model debug DL container OS support: • ARM Android OS Compute runtimes CPU GPU DSP • ARM Linux • x86 Linux Half Fixed Floa
Slide 19: User Defined Layer (UDL) workflow
User Defined Layer (UDL) workflow Supports prototyping of layers not yet supported by the Snapdragon NPE Model conversion tool NPE workflow Model file Model NPE model .dlc file (static weights conversion tool (.dlc file) and biases) NPE runtime Model Conversion Tool Model file NPE workflow (static w
Slide 20: Converting and quantizing a model
Converting and quantizing a model snpe-<framework>-to-dlc snpe-dlc-optimize Model Model Native model NPE model NPE model conversion optimization (e.g., Caffe) (.dlc file) (.dlc file) tool tool Only needed for certain features Images Optional (quantization, NPE enabled app Images (e.g., activation qu
Slide 21: API example usage
API example usage Application NPE new SNPEBuilder ( DLContainer ) NPE provides a simple C++ API SNPEBuilder Object(SBO) with the following functionality SBO->setRunTimeProcessor( RuntimeMode ) Initialization See SNPEBuilder.hpp SBO->setUdlBundle( UdlFactoryContextBundle ) • Load a DLC model and sele
Slide 22: Massive scale: AI + Qualcomm Technologies
100s of Millions AI + Qualcomm Technologies Massive scale of units Learn more at: developer.qualcomm.com Based on Snapdragon Platforms capable of supporting Snapdragon NPE today. 22
Slide 23: Qualcomm Technologies and Facebook collaboration for massive AI scale
Qualcomm Technologies and Facebook collaboration for massive AI scale 23
Slide 24: Facebook + Qualcomm Technologies
Facebook + Qualcomm Technologies On-device AI with Snapdragon Facebook and Qualcomm’s Future Caffe2/NPE “On-device machine learning Caffe2 collaboration research and development is made possible by the • Demonstrated Caffe2 acceleration • Continue to work closely with Qualcomm Snapdragon NPE with NP
Slide 25: Enhancing the Facebook experience through on-device AI
Enhancing the Facebook experience through on-device AI More engaging social media with AI and AR Augmented reality features potentially powered by AI • Style transfer and filters • Frames and masks • Photo and live videos, including 360° • Contextual awareness (e.g. location/sensor metadata) On-devi
Slide 26: What's next
What’s next 26
Slide 27: What does the future look like?
Distributed computing architectures AI hardware What does the future look like? AI acceleration research 27
Slide 28: AI offers enhanced experiences and new capabilities for smartphones
AI offers enhanced experiences and new capabilities for smartphones True personal assistance Superior photography Extended battery life Natural user interfaces Enhanced connectivity Enhanced security A new development paradigm where things repeatedly improve 28
Slide 29: AI will bring XR closer to the ultimate level of immersion
AI will bring XR closer to the ultimate level of immersion Creating physical presence in real or imagined worlds Visuals Interactions Rendering techniques Natural UI Depth estimation Battery efficiency Sounds Managing workload Audio filtering concurrency and cleanup 29
Slide 30: AI is revolutionizing the car of the future
AI is revolutionizing the car of the future Redefining the in-car experience • Natural user interfaces • Personalization • Driver awareness monitoring Paving the road to autonomy • Surround view perception • Sensor fusion • Path planning • Decision making 30 30
Slide 31: What's next?
Reference What’s next? Specialized hardware Algorithmic advancements Improved optimization strategies 31
Slide 32: Thank you!
Thank you! Follow us on: For more information, visit us at: www.qualcomm.com & www.qualcomm.com/blog Nothing in these materials is an offer to sell any of the References in this presentation to “Qualcomm” may mean Qualcomm components or devices referenced herein. Incorporated, Qualcomm Technologies,