Mobileye Global Inc. · NASDAQ:MBLY
Mobileye Global — CES 2021
63 slides from Mobileye Global's conference presentation, January 2021
Slide 1: Mobileye “Under the Hood”
Mobileye “Under the Hood” CES 2021 Prof. Amnon Shashua
Slide 2: Business Pillars
Business Pillars Full Stack ADAS REM™ Mapping Self-Driving System L1- L2++ Advance Driver HD maps for AV, ADAS For consumer AV + Assistance Systems and smart cities Robotaxi MaaS 2
Slide 3: Our ADAS Business in 2020
Our ADAS Business in 2020 EyeQ Shipped (M units): 19.3M 20 19.3 49 RUNNING PROGRAMS 17.5 EyeQs shipped in 2020 Globally across 28 OEMs 15 10% YoY growth despite Covid-19 unprecedented challenges 12.4 10 8.7 37 36.2M 6 NEW DESIGN WINS 5 4.4 Total lifetime volume of the 2.7 new design wins 0 2014 2015
Slide 4: Product Portfolio
Product Portfolio Moving beyond silicon Silicon Silicon + PCB Full stack SDS Full stack SDV Customer-facing VaaS MaaS service EyeQ SoC + embedded Si+PCB integrated Extension of the Si+PCB Based on AV-ready Full stack service ADAS SW solution for L2++ business line platform supplied by Owning the end
Slide 5: The Trinity of Mobileye’s Approach
The Trinity of Mobileye’s Approach The ADAS !" AV divide REM™- enabled scalability RSS formal safety model NOT range of capability, but MTBF AV-Map Key to High MTBF Decision-making governance: TRUE REDUNDANCY™ REM™ Responsibility-Sensitive Safety + AV is a system composed of independent + To make th
Slide 6: The Camera-only Subsystem
The Camera-only Subsystem COMPUTE: SENSOR SETUP: End-to-end operation based on 2x EyeQ5 ① running multiple independent computer vision ② engines for “algorithmic redundancy” ③ ④ ④ CAMERA PARKING CAMERA ① Main 120° ② Narrow 28° ③ Rear 60° ④ Side x4 100° ⑤ Parking x4 192° HW setup in our test vehicle
Slide 7: Achieving the MTBF Goal for L4
Achieving the MTBF Goal for L4 7
Slide 8: Achieving the MTBF Goal for L2+ with Vidar
Achieving the MTBF Goal for L2+ with Vidar 8
Slide 9: SuperVision™
SuperVision™ Productizing AV Camera-only Subsystem to ADAS Value proposition: + Safety: “Vision Zero” next-gen ADAS + Comfort: market-leading L2+ experience First customer: + Geely, Lynk & Co models + SoP: Q4, 2021 Content: + 2x EyeQ 5, PCB, perception, driving policy, REM HD-maps, control, parking
Slide 10: Automotive
Automotive World cct1902 Mobileye and Geely to offer most robust driver assistance features Mobileye signs driver-assistance deal with Geely, one of China's largest privately held auto makers Catherine Shu E E T China's Geely to Feature Mobileye SuperVision for Scalable ADAS Oar mn L\NK&CO GEELY nob
Slide 11: Expanding Footprint of SuperVision™
Expanding Footprint of SuperVision™ ISRAEL DETROIT MUNICH NEXT gg development Main gg Ongoing gg Already completed gg TOKYO site in Jerusalem testing and thousands of AV SHANGHAI Conducting tests development driving hours PARIS across all of Israel Main hub for NYC* customer demonstrations *based on
Slide 12: Achieving the MTBF Goal for L4
Achieving the MTBF Goal for L4 Building an End-to-End AV Capability Based on Radar+LiDAR as an Independent Subsystem Why this is not a trivial problem + Classifying and tracking objects in the absence of texture + Segmenting and tracking objects in cluttered environments + LiDAR-based localization i
Slide 13: Achieving the MTBF Goal for L4
Achieving the MTBF Goal for L4 13
Slide 14: REM™ Maps for ADAS and Autonomous Driving
REM™ Maps for ADAS and Autonomous Driving 14
Slide 15: The Motivation Behind High Resolution Maps
The Motivation Behind High Resolution Maps It is possible to detect and correctly interpret all roadway data, online: • Drivable path • Lane-TFL association • Lane priory • Crosswalks • Path delimiters • Stopping/ yield points But, Making this work at a very high MTBF is not realistic for current AI
Slide 16: The Challenges
The Challenges MAPS AT SCALE UP-TO-DATENESS ACCURACY Robotaxi can sustain lack Time-to-reflect-reality, ideally close Host vehicle and other surrounding road users of scalability (for now) to real time need to be positioned at a cm-level accuracy Consumer-AV needs to drive “everywhere” Lack of scala
Slide 17: The Common Approach for Building HD Maps
The Common Approach for Building HD Maps • Survey fleet with LiDARS, INS and Cameras • Create a geometrical road model using the dense LiDAR & INS output • Use manual / semi-automatic annotation to fine-tune semantic features • Fuse external data layers to enrich map with missing information Mobiley
Slide 18: What’s Wrong with the Typical HD Map Approach? Geometric layer
What’s Wrong with the Typical HD Map Approach? The geometric layer is over specified Cm-level accuracy in global coordinates is NOT needed to support AV 18
Slide 19: What’s Wrong with the Typical HD Map Approach? Semantic layer
What’s Wrong with the Typical HD Map Approach? The richness of the semantic layer is very difficult to automate The semantic layer gives an actionable meaning for each element in the map: DRIVABLE PRIORITY TFL/CROSSWALKS TO STOPPING/ COMMON PATHS LANE ASSOCIATION YIELD POINTS SPEED 19
Slide 20: What’s Wrong with the Typical HD Map Approach?
What’s Wrong with the Typical HD Map Approach? How can the drivable path be determined when there DRIVABLE are no lane marks PATHS 2- w ay st PRIORITY re et TFL/CROSSWALKS TO LANE ASSOCIATION STOPPING/ YIELD POINTS et stre way COMMON 2- SPEED 3-way junction with no lane marks 20
Slide 21: What’s Wrong with the Typical HD Map Approach?
What’s Wrong with the Typical HD Map Approach? DRIVABLE PATHS PRIORITY TFL/CROSSWALKS TO LANE ASSOCIATION STOPPING/ YIELD POINTS COMMON Disambiguating traffic lights priority rules SPEED 21
Slide 22: What’s Wrong with the Typical HD Map Approach?
What’s Wrong with the Typical HD Map Approach? DRIVABLE PATHS PRIORITY TFL/CROSSWALKS TO LANE ASSOCIATION STOPPING/ YIELD POINTS Traffic lights structure and logic vary dramatically from country to country COMMON SPEED 22
Slide 23: What’s Wrong with the Typical HD Map Approach?
What’s Wrong with the Typical HD Map Approach? Determining the optimal stopping DRIVABLE PATHS point in terms of viewing range: • Occlusions may happen from objects that are not part of the map (buildings, fences, Occluded area poles, etc.) PRIORITY • To calculate the optimal stopping point, a map w
Slide 24: What’s Wrong with the Typical HD Map Approach?
What’s Wrong with the Typical HD Map Approach? DRIVABLE PATHS PRIORITY TFL/CROSSWALKS TO LANE ASSOCIATION STOPPING/ YIELD POINTS Common driving speed is one of the main factors to reflect the driving culture and road conditions COMMON SPEED There is no feasible way of calculating common speed 24
Slide 25: “AV Map”, not “HD Map”
Mobileye’s Approach: “AV Map”, not “HD Map” Designing a map that suits exactly what AVs need: Scalability Accuracy Detailed Semantic features Unlocking millions of “mapping Use novel state-of-the-art algorithms Use explicit attributes and “wisdom agents” in every relevant region to achieve high accu
Slide 26: REM™ Under the Hood
REM™ Under the Hood Harvesting Alignment Modeling & Semantics Single drive reports everything Creating a coherent statistical model in a global Creating a definitive, actionable modeling of all it detected coordinates system by aligning random crowd road elements with accurate semantic drives accura
Slide 27: RSD Harvesting
RSD Harvesting Each Harvesting agent collects information on all the relevant road elements Road Geometry • Visual landmarks for localization • Traffic signs & traffic lights • Lane marks & road edges • Crosswalks Semantics • Drivable paths • TSR type • Stopping points • Speed Information is sent as
Slide 28: Aligning Drives
Aligning Drives Aligning random crowdsourced drives to create a 3D model at a cm-level accuracy in the surrounding of the host vehicle. The task is to find the 6D pose (in the map coordinates) for each detected element in each RSD Translate hundreds of random drives that traversed the same location
Slide 29: Why this is hard?
Why this is hard? Every class has its own noise model, thus impossible to implement naïve averaging methods ALIGNMENT BASED ON GPS ZOOM IN Road edge Lane mark Drivable path Sparse lane marks and road edges Poles with different color coding (type) 29
Slide 30: Creating an Extremely Accurate Road Geometry without any Priors
Creating an Extremely Accurate Road Geometry without any Priors THE RESULT: ZOOM IN Road edge Lane mark Drivable path 30
Slide 31: Modeling Process
Modeling Process Clustering and Spline fitting output DNN output DNN for modelling allows to compensate for noisy data and to increase robustness Using “handmade” HD maps as a training set INPUT OUTPUT 31
Slide 32: Why Crowd Sourcing is Perfect for Semantic Understanding
Why Crowd Sourcing is Perfect for Semantic Understanding DRIVABLE PATH IN AREAS WITH DETERMINING ALL POSSIBLE NO LANE MARKS DRIVABLE PATHS IN ADVANCE • Determining where’s the Drivable Path without relying on Lane Marks – • Crowd automatically determines where are the drivable paths, even in imperat
Slide 33: Why Crowd Sourcing is Perfect for Semantic Understanding
Why Crowd Sourcing is Perfect for Semantic Understanding By using crowd observations we can associate instructive road elements with drivable paths in a generalized manner TRAFFIC LIGHT ASSOCIATION YIELD & STOP SIGN ASSOCIATION CROSSWALKS ASSOCIATION Different association to the two drivable paths.
Slide 34: Why Crowd Sourcing is Perfect for Semantic Understanding
Why Crowd Sourcing is Perfect for Semantic Understanding RIGHT OF WAY WHERE THERE ARE WHERE TO STOP DETERMINING YIELD ON GREEN NO TRAFFIC SIGNS TRAFFIC LIGHTS • Relying strictly on traffic signs will result in safety-critical errors where • Crowd behavior provides a robust understanding of where the
Slide 35: Why Crowd Sourcing is Perfect for Semantic Understanding
Why Crowd Sourcing is Perfect for Semantic Understanding COMMON DRIVING SPEED • Provides the target speed in which the AV should drive in when the road is clear. • Understanding how the “average driver” drives helps the AV to blend in naturally, without interrupting traffic. • Crowdsource is the onl
Slide 36: We are Building the World’s Largest Crowdsourcing Fleet
We are Building the World’s Largest Crowdsourcing Fleet Today 2024 7.5B KMS of roads harvested globally 1B KMS of roads covered Daily 8M KMS of roads are covered daily Via 6 major OEM partners and retrofit solution for fleets 36
Slide 37: Active Sensor Development
Active Sensor Development 37
Slide 38: The Motivation Behind LiDAR and Radar Development
The Motivation Behind LiDAR and Radar Development 2022 LiDAR/radar subsystem 2025 LiDAR/radar subsystem • Best-in-class ToF LiDAR (Luminar) - 360° coverage • Front sector- 3-way redundancy ODD case to L5 • Advanced stock radars- 360° coverage • Remaining FoV- 2-way redundancy of cameras + radars mas
Slide 39: SW-defined Imaging Radar Development
SW-defined Imaging Radar Development 39
Slide 40: The Goal
The Goal 1 Increasing angular and vertical resolution 2 Assuring extremely high probability of detection (Pd) by: • Reducing mutual interaction between echoes from different objects • High dynamic range • Very low side lobes levels (SLL) Objects’ side lobes Inherent radar limitations must be overcom
Slide 41: The Required Capabilities
The Required Capabilities Moving from (industry standard): To: To achieve: • Simple MIMO radar with 192 • Massive MIMO radar with 2,304 • High angular and vertical res. (0.5°x2°) virtual channels (12x16) virtual channels (48x48) enabling accurate lane assignment and vertical discrimination • 25dBc a
Slide 42: The Solution: SW-defined Imaging Radar
The Solution: SW-defined Imaging Radar Paradigm shift in architecture to enable the leap in performance: Running on 11 TOPS SoC where usually, the processing Signal processing is shifted in full to the digital domain for of 2,304 channels would require up to 100 TOPS in a superior accuracy non-optim
Slide 43: Slide 43
Slide 44: Detecting two closed pedestrians behind a vehicle
Detecting two closed pedestrians (low RCS) behind a vehicles (high RCS)
Slide 45: Detecting a pedestrian behind parked vehicles
Pedestrian CIPV Park ed ca r Detecting a pedestrian (low RCS) behind parked vehicles (high RCS)
Slide 46: Stable tire detection at 140m
Stable tire detection at 140m 46
Slide 47: FMCW LiDAR Development
n a se ie 7 FMCW LiDAR Development 7 es 47
Slide 48: FMCW LiDAR
FMCW LiDAR Frequency-Modulated EyeC FMCW LiDAR Typ. ToF LiDAR Continuous Wave (FMCW) Angular Resolution 10 LiDAR is conducting a coherent Robustness to 8 Range Resolution Interferences 6 detection and uses Doppler 4 effect to measure both range Night Operation 2 0 Velocity Detection and closing spee
Slide 49: The Required Capabilities
The Required Capabilities Moving from ToF LiDARs To FMCW To achieve: • 3D sampling (range, • 4D (+Doppler for • Instant heading measurement azimuth, elevation) relative velocity • Point-levels TTC measurement) • Reliable clustering and tracking • 200m max. range • 300m max. Range • Enhanced ODD • 1/
Slide 50: Examples
Examples 0.0 (stationary) Vt α β Direct Doppler velocity measurement Vtcos(α) Vtcos(β) • Measurement of TTC at a point level • Reliable clustering and object tracking • Instant heading calculation Velocity (m/s) High Pd at longer ranges FMCW LiDAR can transmit more energy over time (continuous waves
Slide 51: Harnessing Intel’s Si Photonics Leadership to FMCW Sensor Development
Harnessing Intel’s Si Photonics Leadership to FMCW Sensor Development Intel owns the only FAB in the world capable of integrating Active and Passives Components, enabling highly compressed and large scale PIC with the required on-chip interconnects and at high volume manufacturing Optical head enabl
Slide 52: RSS-based Driving Policy
mis em : om \\ \ mie : —_ oa op ne ey, a Driving Pol Sana ees eneencee, 72 | em 4 a rt]
Slide 53: What is RSS? The Method
What is RSS? The Method A formal model for safety, that provides mathematical 01 Defining reasonable boundaries on the behavior of other road users formalization for the AV to never cause an accident Within the boundaries specified by RSS, one 02 must always assume the worst-case behavior of other a
Slide 54: RSS Principles
RSS Principles RSS formalizes five common sense rules apply 01 Do not hit someone from behind to all driving situations 02 Do not cut-in recklessly 03 Right-of-way is given, not taken RSS 04 Be careful of areas with limited visibility 05 If you can avoid an accident without causing another one, you
Slide 55: Industry Standardization Efforts
Industry Standardization Efforts IEEE WORKGROUP TO DEFINE A FORMAL MODEL SAFETY AND CYBERSECURITY FOR FOR AV SAFETY CHAIRED BY INTEL-MOBILEYE AUTOMATED DRIVING SYSTEMS • Workgroup consists of 30 leading industry players • World’s first ISO Technical Report defining a Safe- • 1st Project: Standardizi
Slide 56: Government Efforts
Government Efforts ADVANCED NOTICE OF PROPOSED RULE MAKING: THIRD AND FINAL CONSULTATION, PROPOSING FRAMEWORK FOR AV SYSTEM SAFETY A REGULATORY FRAMEWORK FOR AV’S IN UK • US DOT Seeking public comment on the development of • Builds on three years of work including two previous a framework for Automa
Slide 57: Mobility-as-a-Service
m wa Mobility-as-a-Service _
Slide 58: Mobility Supply is Shaping in Two Main Streams
Mobility Supply is Shaping in Two Main Streams PTOs TNC PRESENT PRESENT Public Transport Transportation Network Operators Companies E.g. RATP, Keolis, Transdev, MTA, etc. E.g. Uber, Lyft, Grab, Didi, etc. APTO ATNC FUTURE FUTURE Automated Public Automated Transportation Transport Operators Network C
Slide 59: Mobility Supply is Shaping in Two Main Streams
Mobility Supply is Shaping in Two Main Streams PTOs TNC PRESENT PRESENT Public Transport Transportation Network Operators Companies E.g. RATP, Keolis, Transdev, MTA, etc. E.g. Uber, Lyft, Grab, Didi, etc. RaaS Through a diversified XaaS VaaS/RaaS strategy Mobileye-Moovit will MaaS serve as the enabl
Slide 60: Harnessing the World’s leading Mobility platform to power our robotaxi service
Harnessing the World’s leading Mobility platform to power our robotaxi service World’s most popular Urban Mobility app • +950 Million users worldwide • Service in 3400 Cities, 112 Countries Largest and most accurate transit data repository • Local Editors Community of over 700,000 volunteers • Used
Slide 61: Mobileye-Moovit Driverless MaaS
Mobileye-Moovit Driverless MaaS The Value of Moovit in our playbook: MaaS Layer 5 UX & UI • Building an E2E service platform (layers 3-5) using Moovit’s assets MaaS Layer 4 Mobility Intelligence and • Pre-acquired public transit User base demand analysis • Global PTO relationship network MaaS Layer
Slide 62: Mobility-as-a-Service Global Footprint
Mobility-as-a-Service Global Footprint First MaaS deployment in Israel Collaboration with WILLER Group • Israel as a sandbox- JV with VW • Targeting commercial launch in Osaka • Early riders program to start 2021 in 2023 • Commercial driverless pilot to start in • Future expansion to Singapore and 2
Slide 63: Thank you!
mn mobileye An Intel Company Thank you! 63