Slide 1: NVIDIA
© NVIDIA
Slide 2: K YO
K YO
Slide 3: NVIDIA Fights COVID-19
NVIDIA FIGHTS COVID-19 Oxford Nanopore Plotly, NVIDIA ORNL, Scripps Structura, NIH, UT Austin NIH, NVIDIA Kiwibot Whiteboard Coordinator Sequence Virus Genome Real-Time Screen CryoSPARC AI COVID-19 Robot Medical Supply Al Elevated Body Temp in 7Hrs Infection Rate Analysis 18 Drug Compounds in 13D St
Slide 4: Computing for the Da Vincis of Our Time
COMPUTING FOR THE DA VINCIS OF OUR TIME
Slide 5: ———— Ss
———— Ss
Slide 6: Performance, Installed Base, One Architecture, Full Stack
PERFORMANCE ++ INSTALLED BASE ++ cUDA PU ONE ARCHITECTURE SYSTEMS FULL STACK X-FACTOR SPEED-UP
Slide 7: X-Factor Speed-Up
X-FACTOR SPEED-UP cUDA FULL STACK DATA CENTER SCALE DEVELOPERS ++ O; INSTALLED BASE ++ ONE ARCHITECTURE PERFORMANCE ++
Slide 8: NVIDIA and Mellanox Technologies
> NVIDIA. Mellanox: TECHNOLOGIES a
Slide 9: Fantastic Year for Our Developers
DISS 20 RIXGI RTX Broadcast Engine JetPack 4 RAPIDS 0.13 MONAL MAGNUM 10 ‘cuTENSOR 1.1 ‘cUDNN 8 TensorRT 7.1 cua tt FANTASTIC YEAR FOR OUR DEVELOPERS ma, wale 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 qe
Slide 10: NVIDIA announced RTX at SIGGRAPH 2018
On August 13, 2018 at SIGGRAPH, NVIDIA announced RTX, the beginning of a new era in computer graphics. “Real-time ray tracing, which had been promised for over . 30 years, arrived a decade earlier than expected.” ~ Jon Peddie, JPR
Slide 11: Ray Tracing and Deep Learning
Ray Tracing QP AA IAL = li WW SOY A Oa ~ REAR OY oN Deep Learning
Slide 12: NVIDIA RTX: Ray Tracing and AI
“a NVIDIA RTX A NEW ERA IN COMPUTER GRAPHICS — RAY TRACING AND Al E 1080p Ray Tracing Supercomputer Rendered - 16K Ground Truth
Slide 13: NVIDIA RTX: Ray Tracing and Deep Learning
NVIDIA RTX A NEW ERA IN COMPUTER GRAPHICS — RAY TRACING AND Al Ray Tracing Deep Learning 4 a
Slide 14: Slide 14
Slide 15: Slide 15
Slide 16: Slide 16
Slide 17: Slide 17
Slide 18: Slide 18
Slide 19: Slide 19
Slide 20: Slide 20
Slide 21: Slide 21
Slide 22: Minecraft
my MINECRAEY
Slide 23: Minecraft RTX Reviews
“Gamechanger”™ Digital Found “My God, it’s gorgeous” CWorld “Nothing short of awesome” “Stunning” = PC Game “Jaw-dropping" ~ Trusted Reviews
Slide 24: Slide 24
Slide 25: Cinematic Production Challenges
CINESS Large Teams of Experts Multiple Locations and Studios technicolor Expensive “Hi aniuattosie DIGITAL mM =J DOMAIN _ wela
Slide 26: Slide 26
Slide 27: Slide 27
Slide 28: Slide 28
Slide 29: Quadro Virtual Workstations and RTX Server
Quadro Virtual Worksta' tions Validated with Design and Etter NVIDIA RTX Server Shipping Now BOX =| HewiettPackard — Gremicye Enterprise
Slide 30: NVIDIA HPC
NVIDIA HPC Chroma x GROMACS hate Quantum Espresso Random Forest System Memory ” TensorFlow SEP= MPAS, PyTorch = peu Clima VASP , | , Molecular Dynamics Amber toca, S38 MiLC 4x Ni NAMD 2x aa x x ks) HPE Apollo70 GROMACS VASP Marvell ThunderX Molaeular Dynamics Material Science ojo) von arm) Vion) ARM
Slide 31: Machine Learning Pipeline Is an HPC Challenge
MACHINE LEARNING PIPELINE IS AN HPC CHALLENGE ETL TRAINING INFERENCE a of10) par oI PyTorch TT 4 a User Query Es eae [____ + Magnum 10 Data Lake 100's PB
Slide 32: Machine Learning Driving Exponential Growth in Data
MACHINE LEARNING DRIVING EXPONENTIAL GROWTH IN DATA ETL TRAINING INFERENCE OlPetal gow < Otteral TensorFlow Cee tines Soar! . PyTorch ¢ Data Set Data Processing fequirement USERS EMBEDDING ; [accor] [a [rrr oft0) Recommendation > L___, + Data Lake 100's PB
Slide 33: NVIDIA Accelerates Spark 3.0
el ANNOUNCING NVIDIA ACCELERATES SPARK 3.0 OlPetal -— oft0) Recommendation > ETL TRAINING INFERENCE an S ark TensorFlow PyTorch [rrr L—___, User Query ee, + Data Lake 100's PB
Slide 34: RAPIDS Shatters ETL Benchmark
“SPARK 3.0 BUI RAPIDS SHATTERS ETL BENCHMARK $1M 17GB/s 17 GB/s Throughput on TPCx-BB@ SF 10K $1M | 182U CPU Systems | 2Racks | 16 kW OlPetal -— DATION ETL TRAINING INFERENCE =< iQ ottera} Teneerelow, Recommendation Soar! _ “4 PyTorch ¥ 4 User Query pL [rrr i+ Penh Se Triton Infe! Data Lake 100's PB
Slide 35: RAPIDS Shatters ETL Benchmark
F ND RAPIDS SHATTERS ETL BENCHMARK a Lt pli TRAINING INFERENCE : | (10) OlPetal soGE Oleral ayaa oxtog) Recommendation >| | enooaes| a | Spark. |“) terete [rrr in a Pm | pes cu USERS EMBEDDING $2M 163 GB/s < 163 GB/s Throughput on RAPIDS Implementation of TPCx-BB@ SF 10K Data Lake 100's PB $2M |16DG
Slide 36: RAPIDS Shatters ETL Benchmark
RAPIDS SHATTERS ETL BENCHMARK ni TRAINING INFERENCE USERS EMBEDDING | (10) OlPetal soGE Oleral ayaa oxtog) Recommendation >| | enooaes| a | Spark. |“) terete [rrr in a Pm | Magnum IO Triton Infer PRICE-PERF t ——___> f 4 < 163 GB/s Throughput on RAPIDS Implementation of TPCx-BB@ SF 10K Data Lake 100'
Slide 37: Spark 3.0 Built on State-of-the-Art Foundation
SPARK 3.0 BUILT ON STATE-OF-THE-ART FOUNDATION RAPIDS SHATTERS ETL BENCHMARK 128 it TRAINING INFERENCE Spark. |=) "eek OlPetal -— USERS EMBEDOING o(t0 . | + a Magnum IO $10M 140 kW I} Recommendation > User Query a Equivalent 163 GB/s Throughput on TPCx-BB @ SF 10K Data Lake 100's PB $10M | 167 2U CP
Slide 38: RAPIDS Shatters ETL Benchmark
F ND RAPIDS SHATTERS ETL BENCHMARK a Lt pli TRAINING INFERENCE : | (10) OlPetal soGE Oleral ayaa oxtog) Recommendation >| | enooaes| a | Spark. |“) terete [rrr in a Pm | USERS EMBEDDING 1/5th 1/34 cosT POWER t ——___> = 4 < 163 GB/s Throughput on RAPIDS Implementation of TPCx-BB@ SF 10K Data Lake 100
Slide 39: Databricks Accelerated with NVIDIA
ANNOUNCING DATABRICKS ACCELERATED WITH NVIDIA ETL TRAINING INFERENCE Y = spark’ =) Tere” |—less|2* ‘ databricks a [_resenaone | These contributions lead to faster data oft0} Recommendation > User Query ee, pipelines, model training and scori more breakthroughs and insights w hy Treen Apache Spark 3.
Slide 40: Cloud Analytics Platforms Accelerated with NVIDIA
ANNOUNCING CLOUD ANALYTICS PLATFORMS ACCELERATED WITH NVIDIA ETL TRAINING INFERENCE — otto} 11009) | Recommendation anooates| 11081 gay ol a User Query [aoc] |e APACHE K¢ Olteral TensorFlow OlPeta) Si econ = |, Sorers.<: |-St)> ae , databricks Google Cloud Al Magnum 10 Magnum 10 Google Cloud Datapro
Slide 41: NVIDIA AI
NVIDIA Al ETL TRAINING INFERENCE USERS EMBEDDING PyTorch “a pond iN, spank’ pial TensorFlow i cuONN ‘CUDA-X [ee [rrr
Slide 42: Recommender System Pipeline
| Item Embedding (107) Herr | eae User Embedding Candidate Generation ol10) Recommended tems
Slide 43: NVIDIA Merlin Deep Recommender Framework
ANNOUNCING NVIDIA MERLIN — DEEP RECOMMENDER APPLICATION FRAMEWORK 1TB Ads Dataset TRAINING 1 % Days TRAINING 16 Min NVIDIA Merlin NVTabular RAPIDS HugeCTR o(10 Triton Inference Server I} Recommendation > User Query ee, + Data Lake 100's PB
Slide 44: TensorRT AI Inference
TENSORRT 7 © NVIDIA Al INFERENCE 15X TENSORRT DOWNLOADS IN 2 YEARS 15X DOWNLOADS €2 aws Balam Alba con Ss © criteol. ebay {FO Google Cloud 10X DEVELOPERS BSD NAVER San za za ® &
Slide 45: NVIDIA Jarvis Conversational AI Framework
ANNOUNCING NVIDIA JARVIS — MULTIMODAL CONVERSATIONAL Al SERVICES FRAMEWORK Gi anIcing Omniverse
Slide 46: Slide 46
Slide 47: Jarvis Demo
JARVIS DEMO MISTY — INTERACTIVE 3D CHATBOT Gi anIcing Omniverse
Slide 48: we
we
Slide 49: NVIDIA Jarvis Conversational AI Framework
ANNOUNCING NVIDIA JARVIS — MULTIMODAL CONVERSATIONAL Al SERVICES FRAMEWORK NVIDIA JARVIS TRITON INFERENCE SERVER
Slide 50: NVIDIA Jarvis Conversational AI Framework
ANNOUNCING NVIDIA JARVIS — MULTIMODAL CONVERSATIONAL Al SERVICES FRAMEWORK PRE-TRAINED MODEL RE-TRAIN NVIDIA JARVIS NVIDIA GPU CLOUD NVIDIA Al TOOLKIT Join Early Access Program developernvidia.com/nvidia-jarvis TRITON INFERENCE SERVER
Slide 51: Conversational AI Is Transforming Industries
CONVERSATIONAL Al IS TRANSFORMING INDUSTRIES VIDEOCONFERENCE CALL CENTER SMART SPEAKERS RETAIL ASSISTANTS. IN-CAR ASSISTANTS CC, TRANSLATION, TRANSCRIPTION 500M Calls per Day 150M Sold per Year 12M Retail Stores 75M New Cars per Year 200M Meetings per Day UN Ne. ‘ (2 S&P Global EE Microsoft voca.a f
Slide 52: CUDA-X
‘CUDA-X ‘MAGNUMIO =i cuONN NVIDIA Al 2. NVIDIA Merlin @@ NVIDIA Jarvis Ft TensorFlow [rrr Triton Inference Server
Slide 53: Modern Cloud Data Center
MODERN CLOUD DATA CENTER Diverse Applications | Scale-Up & Scale-Out Workloads | Insatiable Demand
Slide 54: Announcing NVIDIA A100
ANNOUNCING NVIDIA A100 54 BILLION XTORS SROGEN TENSOR CORES SPARSITY ACCELERATION MiG 3° GEN NVLINK & NVSWITCH
Slide 55: Announcing NVIDIA A100
ANNOUNCING NVIDIA A100 TSMC 7nm | HBM2— 1.6 Terabytes per Second | 3D Chip Stack 54 BILLION XTORS SROGEN TENSOR CORES SPARSITY ACCELERATION MiG 3° GEN NVLINK & NVSWITCH
Slide 56: New TF32 Tensor Cores
——————————————— NEW TF32 TENSOR CORES Range of FP32 and Precision of FP16 | Input in FP32 and Accumulation in FP32 | No Code Change Speed-up for Training TENSOR FLOAT32
Slide 57: TF32 Tensor Cores
RES Range of FP32 and Precision of FP16 | Input in FP32 and Accumulation in FP32 | No Code Change Speed-up for Training NVIDIA V100 FP32 NVIDIA A100 Tensor Core TF32
Slide 58: New Tensor Core Acceleration for Sparsity
Ed NEW TENSOR CORE ACCELERATION FOR SPARSITY Optimized For Sparse Al Tensor Ops| 2X Faster Execution | Supported on TF32, FP16, BFLOAT16, INT8 and INT4 Ce0S SxS 2X OSOR29 CROSSE Effective CPOSS CONS 7 i] , . a > i es a — Dense Matrix Sparse Matrix A100 Sparsity Optimized Tensor Core
Slide 59: Announcing NVIDIA A100: Greatest Generational Leap
EE —————————— ANNOUNCING NVIDIA A100 GREATEST GENERATIONAL LEAP as = = vi00 vi00 vi00 tae tia inte
Slide 60: Announcing NVIDIA A100: Greatest Generational Leap
Ed ANNOUNCING NVIDIA A100 GREATEST GENERATIONAL LEAP 05 a0 to 12! it | lussal oo vio oo vioo aoa vio aoa trae Ta 1 Pte nts Nt
Slide 61: Announcing NVIDIA A100: Greatest Generational Leap
v100 FP32 ‘A100 TF32 310 310 125 il 60 Le! A100 viod A100 A100 viog A100 Ato, SPARSE FPI6 FPI6 SPARSE INTE INTS SPARS! 1F32 FPIB IN — ANNOUNCING NVIDIA A100 GREATEST GENERATIONAL LEAP
Slide 62: Announcing NVIDIA A100: Greatest Generational Leap
— ANNOUNCING NVIDIA A100 GREATEST GENERATIONAL LEAP 310 1250 20x ' — 025 20 5310 8 on 16 e al n -_ -_ _— Le! ‘Aloo A100 viod 100 A100 vioo A100 _ato0 FP64 FPGA FP32 1F32 SPARSE FPI6 FPI6 SPARSE INTE INTS SPARSE 1F32 FPIB INTS
Slide 63: Multi-Instance GPU Computing
PU COMPUTING 7x Higher Throughput of V100 with Simultaneous Instances per GPU
Slide 64: BERT Training and Inference Throughput
BERT Pre-Training Throughput using Pytorch including (2/3)Phase 1 and (1/3)Phase 2 | Phase 1 Seq Len = 128, Phase 2 Seq Len = 512 V100: DGX-1 Server with 8xV100 using FP32 precision A100: DGX A100 Server with 8xA100 using TF32 precision | BERT Large Inference | T4, V100: TRT 7.1, Precision = FP16, B
Slide 65: Batch One Inference Performance
= —_— — “<7 \Batch One Inference Performance Audio Classification and BERT Question and/ Answer arr i north america
Slide 66: Announcing NVIDIA A100
ANNOUNCING NVIDIA A100 54 BILLION XTORS SROGEN TENSOR CORES SPARSITY ACCELERATION MiG 3° GEN NVLINK & NVSWITCH
Slide 67: PetaFLOPS of Performance in a Single Node
5 PetaFLOPS of Performance in a Single Node Unified System for End-to-End Data Science and Al Fully Accelerated Stacks — Spark 3.0, RAPIDS, TensorFlow, PyTorch, Triton Elastic Scale-Up or Scale-Out Computing High Scalability with Mellanox Networking
Slide 68: Announcing NVIDIA DGX A100
ANNOUNCING NVIDIA DGX A100 382 GENERATION INTEGRATED Al SYSTEM 5 PetaFLOPS of Performance in a Single Node
Slide 69: Announcing NVIDIA DGX A100
ANNOUNCING NVIDIA DGX A100 382 GENERATION INTEGRATED Al SYSTEM 5 PetaFLOPS of Performance in a Single Node INT8 10 PetaOPS Peak FP16 5 PFLOPS Peak TF32 2.5 PFLOPS Peak FP64 156 TFLOPS Peak TensorCore with Sparsity
Slide 70: Announcing NVIDIA DGX A100
ANNOUNCING NVIDIA DGX A100 380 GENERATION INTEGRATED Al SYSTEM 5 PetaFLOPS of Performance in a Single Node 150X Al Compute 40X = Memory Bandwidth 40X 10 Bandwidth Compared to High-End CPU server
Slide 71: DGX A100 Available Now at $199K
ANNOUNCING NVIDIA DGX A100 30 GENERATION INTEGRATED Al SYSTEM 5 PetaFLOPS of Performance in a Single Node 150X Al Compute 40X = Memory Bandwidth 40X 10 Bandwidth Compared to High-End CPU server Available Now at $199K
Slide 72: DGX A100 Cloud and System Partners
©) Alibaba Cloud aws (BAIDU AICLOUD Sad i ORACLE Z @ FP Microsoft Azure Cloud infrastructure a Google Cloud ‘Cows SYSTEMS Hewlett Packard Lenovo. Enterprise
Slide 73: Today's AI Data Center
TODAY'S Al DATA CENTER 50 DGX-1 Systems for Al training 600 CPU Systems for Al Inference $11M 25 Racks 630 kW $11M 630 kW
Slide 74: DGX A100 AI Data Center
DGX A100 Al 5 DGX A100 Systems for Al Training and Inference $1M pee $1M 28kW 28 kW 1/10% ELE ISS Uiiinos
Slide 75: Today's AI Data Center
TODAY'S Al DATA CENTER 50 DGX-1 Systems for Al training 600 CPU Systems for Al Inference $11M 25 Racks 630 kW
Slide 76: DGX A100 AI Data Center
DGX A100 Al 5 DGX A100 Systems for Al Training and Inference $1M 1 Rack 28 kW LES Mii
Slide 77: PageRank Case Study
PAGERANK CASE STUDY Common Crawl Data Set 2.61B Graph - 128B Edge 3,000 CPU Servers - 105 Racks
Slide 78: PageRank Case Study
PAGERANK CASE STUDY Common Crawl Data Set 2.6TB Graph - 128B Edge 4 DGX A100 Connected via External NVLINK 688 Billion Edges / Sec 13X PERFORMANCE Ce
Slide 79: PageRank Case Study
PAGERANK CASE STUDY Common Crawl Data Set 2.61B Graph - 128B Edge 3,000 CPU Servers - 105 Racks
Slide 80: PageRank Case Study
PAGERANK CASE STUDY Common Crawl Data Set 2.6TB Graph - 128B Edge 4 DGX A100 Connected via External NVLINK Ll
Slide 81: Announcing NVIDIA DGX A100 SuperPOD
ANNOUNCING NVIDIA DGX A100 SUPERPOD 140 DGX A100 Systems (1,120 A100) 170 Mellanox Quantum 2006 InfiniBand Switches 280 Tb/s Network Fabric - 15km of Optical Cable 4 PB of All-Flash Networked Storage 700 PFLOPS of Al Performance Built in under 3 Weeks
Slide 82: Slide 82
Slide 83: Slide 83
Slide 84: Slide 84
Slide 85: Slide 85
Slide 86: Slide 86
Slide 87: Slide 87
Slide 88: Slide 88
Slide 89: Slide 89
Slide 90: Slide 90
Slide 91: +
+
Slide 92: DRIVE Platform from ADAS to Robotaxi
ADAS L2+ L5 Windshield NCAP Autopilot Robotaxi 10 TOPS, 5W 200 TOPS, 45W 2,000 TOPS, 800W
Slide 93: Slide 93
Slide 94: Slide 94
Slide 95: Slide 95
Slide 96: Slide 96
Slide 97: Slide 97