Nvidia Corporation · NASDAQ:NVDA
Nvidia — New Street Big Ideas in Semiconductors Conference 2021
12 slides from Nvidia's conference presentation, Sep 9, 2021
Slide 1: Scaling AI and Using It
SCALING AI AND USING IT Bryan Catanzaro; September 9, 2021
Slide 2: Forward-looking statements
Forward-looking statements Except for the historical information contained herein, certain matters in this presentation including, but not limited to, statements as to: language model complexity doubling every two months; expected timing for 100T parameter single model; benefits, cost and impact of
Slide 3: Large Language Models: Exploding Model Complexity
LARGE LANGUAGE MODELS Exploding Model Complexity Doubling Every 2 Months Compute required to train 175B OpenAI GPT-3 1.E+04 GPT-3 314 ZettaFLOP for training (3640 PFLOP/s * day) 1.E+03 Megatron-BERT Petaflop/s - Days Megatron-GPT2 Turing NLG 1.E+02 GPT-2 PetaFLOPS - Days 1.E+01 BERT 1.E+00 ResNet 1.
Slide 4: Towards the $1B Model
TOWARDS THE $1B MODEL The value of few-shot learning English: I live in California. Spanish: Yo vivo en California. These models are expensive (GPT-3 cost $12M to train) English: I work at NVIDIA. But their value is great Spanish: Yo trabajo en NVIDIA. English: I believe in science. Generalized inte
Slide 5: Megatron Scaling on DGX SuperPOD
Hidden Number of Model Number Case MEGATRON SCALING Size Layers Parallel Size of GPUs 1T 25600 128 512 3072 ON DGX SUPERPOD 530B 20480 105 280 2520 145B 12288 80 64 1536 39B 8192 48 16 512 All of NVIDIA’s HW and SW working together 7.5B 4096 36 4 128 CUDA, CUDA-X AI, NVSwitch, DGX SuperPod, NCCL, CU
Slide 6: NVIDIA DLSS
NVIDIA DLSS Reinventing a Core NVIDIA Business with Deep Learning 4MPixel (1440P) VS 8MPixel (4K) Ray Tracing Supercomputer Rendered 128MPixel (16K) Ground Truth Deep Learning Super Sampling 6 @ctnzr
Slide 7: Native 4K vs DLSS 4K
108 141 NATIVE 4K DLSS 4K 7
Slide 8: Native 4K, DLSS Off and DLSS On
60 108 100 141 NATIVE 4K DLSS OFF DLSS ON DLSS 4K 8
Slide 9: DLSS reviewer quotes
“ DLSS is impressive to the point where I believe you'd be nuts not to use it.” — Digital Foundry “ The upscaling power of this new AI driven algorithm is extremely impressive… it’s basically a free performance button.” — Hardware Unboxed 9
Slide 10: Creating a Self-Driving Car Platform, Infrastructure, and Service
HD Map CREATING A SELF-DRIVING Map CAR PLATFORM, INFRASTRUCTURE, Map Generation AND SERVICE Data Model Data Factory Synthetic Data Model Perception Model AI Training Data Data Model Drive Sim Data Driving AI Training AV DRIVE AV 10
Slide 11: Scaling AI and Using It
SCALING AI AND USING IT AI is growing at a super-exponential rate NVIDIA is building the software and systems to create the most important AI We are applying advanced AI to transform and grow our core business We offer the tools for other companies to do so as well 11
Slide 12: < Q =
< Q 5 =
NVIDIA — New Street Big Ideas in Semiconductors Conference 2021 on 6ix