Slide 1: The mobile future of augmented reality
Oct 2018 @qualcomm_tech The mobile future of augmented reality Qualcomm Technologies, Inc.
Slide 2: Agenda
Agenda 1 Brief introduction to Augmented Reality 2 Evolution of AR from today to the future 3 New technologies for AR requirements 2
Slide 3: Augmented reality will impact every aspect of our lives
Augmented reality will impact every aspect of our lives Offering unprecedented experiences and increased productivity 3
Slide 4: AR and VR offer distinct experiences
AR and VR offer distinct experiences But share similar underlying technologies Virtual reality Augmented reality Simulates physical presence in real or imagined worlds, Superimposes content over the real world such that the and enables the user to interact in that world content appears to be part of
Slide 5: Evolution of user experience from VR to AR
Evolution of user experience from VR to AR Today Soon The future VR: Mostly 3-DOF, lower resolution videos VR: Ability to move around through live XR: Entire scenes, like entertainment and games events, with better sense of “presence” events, can be accessed with your mobile AR device that are so re
Slide 6: AR will transform the way we live and interact with the world
AR will transform the way we live and interact with the world Offering unprecedented experiences and increased productivity A plurality of experiences in AR Play Explore Communicate Work Thrive Interactive gaming in Immersive education Telepresence Training and productivity Health any environment an
Slide 7: AR will serve a broad spectrum of roles in daily life
AR will serve a broad spectrum of roles in daily life Applicable across ages, genders, and activities Children Young Families Professionals Fitness Enthusiasts Playing Adults Exploring Communicating Working Thriving Kids chasing virtual A young man exploring Families virtually Architects collaborati
Slide 8: AR will revolutionize industries and enterprises
AR will revolutionize industries and enterprises Increased productivity, efficiency, and safety Industrial and manufacturing Engineering • Guided training and remote support • 3D visualization and CAD • Improved safety • Colleague collaboration and • Real-time factory diagnostics communication Healt
Slide 9: AR is the next mobile computing platform
AR is the next mobile computing platform Nearly everything we’ve learned for smartphones will be used for AR AR glasses Smartphone Laptop 9
Slide 10: AR technologies and use cases evolve from mobile
AR technologies and use cases evolve from mobile VR usage primarily comes from console/TV/PC, but it’s also moving towards AR Ultimately, this becomes an imperceptible Landline PDA device that replaces nearly all others Handheld Camera gaming AR glasses Laptop Smartphone VR HMD TV Desktop PC Gaming
Slide 11: AR is here today, but it is still in its infancy
AR is here today, but it is still in its infancy Like smartphones, the AR evolution will take years but has the potential to be huge Technology Phase: Infancy Market: Mostly early adopter “Prosumers” Technology Phase: Rapid evolution Market: Surging consumer adoption Technology Phase: Maturity Marke
Slide 12: A glimpse into the future
A glimpse into the future First responder AR glasses Bone conduction transducers Multimode connectivity (4G, 5G, etc.) Rear camera • Many passive and active cameras with fisheye and telephoto lenses • Optoelectronic night vision and thermal imaging sensors Ambient light sensors Directional speakers
Slide 13: AR will seamlessly merge the real world with virtual objects
AR will seamlessly merge the real world with virtual objects Providing an always-on experience that intelligently enhances our lives Immersive Intelligent Connected The visuals, sounds, and interactions are It understands the real world, learns personal An always-on, low power wearable with fast so
Slide 14: Immersive
Immersive The visuals, sounds, and interactions are so realistic that they are true to life 14
Slide 15: AR shares requirements similar to VR for immersion
AR shares requirements similar to VR for immersion Achieving realistic AR at low power to enable a comfortable, sleek form factor Extreme pixel quantity and quality Screen is very close to the eyes High resolution audio Up to human hearing capabilities Spherical view Visual Sound Look anywhere with
Slide 16: AR introduces additional requirements for immersion
AR introduces additional requirements for immersion Seamlessly integrating virtual objects with the real world Keeping the world stable Seamlessly anchor virtual objects to the real world Visual Sound Realistic Virtual Sounds Occlusion quality Modifying virtual sounds based Showing and hiding virtua
Slide 17: Keeping the world stable
Keeping the world stable In an unstable environment, virtual objects are NOT seamlessly anchored to the real world 17
Slide 18: Keeping the world stable
Keeping the world stable In a stable environment, virtual objects are seamlessly anchored to the real world 18
Slide 19: Occluding virtual objects correctly
Occluding virtual objects correctly Incorrect occlusion breaks immersion 19
Slide 20: Occluding virtual objects correctly
Occluding virtual objects correctly Correct occlusion accounts for the depth of virtual and real objects 20
Slide 21: Occluding virtual objects correctly
Occluding virtual objects correctly Smart occlusion accounts for both object depth and user preferences 21
Slide 22: Lighting virtual objects realistically and dynamically
Lighting virtual objects realistically and dynamically Incorrect lighting poorly represents the position, intensity, and orientation of all light sources Poor environment processing • Virtual objects look fake and out of place • Static lighting; often incorrect for environment • Solid objects do not
Slide 23: Lighting virtual objects realistically and dynamically
Lighting virtual objects realistically and dynamically Correct lighting considers the position, intensity, and orientation of all light sources Proper AR environment processing • Virtual objects look real and correctly placed • Dynamic lighting; correct for the environment • Solid objects look solid
Slide 24: Creating virtual sounds based on the real world
Creating virtual sounds based on the real world Sound reflections spread and interact with the environment appropriately Airport • Limited sound reflections Hotel room • Significant ambient sound • Significant sound dampening Conference room • Enclosed room with reflective surfaces Required • Virtua
Slide 25: Interacting naturally with AR
Interacting naturally with AR Interactions will become more intuitive and adaptive to personal preferences Motion and gesture recognition Use CV along with motion sensors, and new types of connected, haptic devices to help users interact within AR Speech recognition and learning Face recognition Use
Slide 26: Intelligence
Intelligence Understanding the real world and learning personal preferences 26
Slide 27: AI technologies are key for AR adoption
AI technologies are key for AR adoption Making sense of the world while protecting our privacy and security Machine learning Security and privacy are critical for AR Makes visual, audio, Continuous Utilize combined machine and other sensor processing authentication necessary learning on biometrics a
Slide 28: Intelligent AR will greatly expand our human abilities
Intelligent AR will greatly expand our human abilities By understanding the environment and providing personalized assistance Make travel easier Assist the visually impaired Become a pro Describe the landmarks around you Help the visually impaired map their Make a gourmet meal, fix your car, and tra
Slide 29: Contextual intelligence to "sense" the world
Contextual intelligence to “sense” the world The fusion of many types of sensors and personal information is required Sensor data On-device data Off-device data Environment Iris scan Ambient light Compass Calendar Cloud data Camera Microphone Temperature Humidity Messaging IoT data Sensor fusion Gyr
Slide 30: Visual intelligence to "see" the world around you
Visual intelligence to “see” the world around you Continually monitoring the visual world to intelligently identify 3D depth capture, interpolation, and 29 feet reconstruction Using passive + active cameras, along with advanced CV and 11 feet machine learning algorithms Object recognition, tracking,
Slide 31: Recognize, track, map and reconstruct surroundings
Recognize, track, map amd reconstruct surroundings Underlying technologies Object recognition, Simultaneous localization Visual inertial 3D reconstruction tracking and registration and mapping (SLAM) odometry (VIO) 31
Slide 32: Understand and inform
Lisbon, Portugal Understand and inform Temp: 98° F Recognize the relationship between objects and provide relevant information Cable Car Cost: €3 Departing: 10 min 18th Street Destination time: 1 hr Café 4.5 stars Serves Caldo Verde Taxi Sarah M. Cost: €25 Last talked: 2 months Departing: Now Ask ab
Slide 33: Recommend actions
Recommend actions Personalized virtual tour guide “Catch this trolley to take your next tour. This one also has air conditioning available. Have €3 ready.” 33
Slide 34: AI to autonomously take actions
AI to autonomously take actions Infer context and anticipate needs Autonomously take predictive, necessary actions High level concept understanding with learned perception and awareness Motion User identification Ambient speech and Personal places and Object detection Cloud data classification and p
Slide 35: Connected
Connected An always-on device with fast wireless cloud connectivity anywhere 35
Slide 36: AR requires the next level of ubiquitous connectivity
AR requires the next level of ubiquitous connectivity The mass adoption of a highly mobile, intelligent, always-on wearable device Providing extreme Connecting Enabling cloud capacity everything computing 36
Slide 37: AR drives the need to efficiently grow wireless capacity
AR drives the need to efficiently grow wireless capacity High-capacity connectivity to share and consume content 24/7 Downlink/ Demanding and generating more data Consume Visual content, such as videos,images, or graphics • Higher resolution, higher frame rate, stereoscopic, High Dynamic Range (HDR)
Slide 38: Gigabit Class LTE is the foundation for AR wireless capacity
Gigabit Class LTE is the foundation for AR wireless capacity Technology enablers for achieving Gigabit Class LTE Carrier Aggregation Combines multiples LTE carriers for wider bandwidth, e.g. fatter pipe Aggregation across diverse spectrum types More bandwidth Makes the best use of spectrum, e.g. FDD
Slide 39: Contextual intelligence requires connecting everything
Contextual intelligence requires connecting everything UI augmentation is limited without hyper-connectivity Creating a Connecting: connectivity fabric • Device to device for everything • Device to cloud Securely gathering contextual data for • AR glasses to devices increased intelligence • AR glass
Slide 40: Cloud computing complements on-device processing
Cloud computing complements on-device processing Leveraging ubiquitous connectivity for the best of both worlds On-device processing Cloud computing Security and Boundless cloud data user privacy and storage Both are needed Low latency for instant Crowd-sourcing and processing data aggregation Avail
Slide 41: Taking AR experiences to the next level with 5G
Taking AR experiences to the next level with 5G Ubiquitous coverage with Gigabit LTE / 5G multi-mode devices Enjoy AR experiences everywhere Share real-time/interactive experiences At home, at work, at school, in the car, walking around… Events, meetings, telepresence… Extreme throughput Ultra-low l
Slide 42: Power efficiency is essential for AR
Power efficiency is essential for AR The AR headset needs to be comfortable to wear all day Constrained mobile AR workloads wearable environment • Compute intensive • Thermally efficient for sleek, ultra-light designs • Complex concurrencies • Long battery life for • Always-on all-day use • Real-tim
Slide 43: QTI is uniquely positioned to support superior AR experiences
QTI is uniquely positioned to support superior AR experiences Custom designed SoCs and investments in the core AR technologies 43
Slide 44: We're developing foundational technology for AR
We’re developing foundational technology for AR Qualcomm Technologies’ investments and the confluence of mobile technologies Computer AI Heterogenous Next-gen vision and security computing connectivity • 6-DOF VIO • AI for advanced • Lower power, higher • Gigabit LTE cognitive processing perf. AR vi
Slide 45: Snapdragon is designed to meet AR requirements
Snapdragon 845 is designed to meet AR requirements Utilizing specialized engines across the SoC for efficient processing LPDDR4X Qualcomm Qualcomm® Memory Spectra™ Snapdragon™ X20 280 ISP LTE Modem Augmented reality requires heterogeneous computing Display Processor Qualcomm® Hexagon™ Video Processo
Slide 46: We are also investing in these innovative start-ups
We are also investing in these innovative start-ups Qualcomm Ventures portfolio Professional “light field” Professional 3D reconstruction Smartphone AR software for cameras and software cameras and software “visual marketing” Software & hardware Virtual reality game studio Wearable mixed reality for
Slide 47: AR is the next mobile computing platform
AR is the next mobile computing platform AR is here today, but still in infancy Advancements are required to make AR optimally immersive, intelligent, and connected Qualcomm Technologies will continue to innovate AR technologies 47
Slide 48: Thank you
Thank you Follow us on: For more information, visit us at: www.qualcomm.com & www.qualcomm.com/blog Nothing in these materials is an offer to sell any of the References in this presentation to “Qualcomm” may mean Qualcomm components or devices referenced herein. Incorporated, Qualcomm Technologies,
Slide 49: Resources
Resources • Websites ◦ Augmented reality: https://www.qualcomm.com/AR ◦ Virtual reality: https://www.qualcomm.com/VR ◦ Immersive experiences: https://www.qualcomm.com/Immersive ◦ Developers: https://developer.qualcomm.com ◦ Newsletter signup: http://www.qualcomm.com/mobile-computing-newsletter • Pre