Slide 1: Forward-looking statements
This event includes forward-looking statements about future products and other topics, which are based on our current expectations and subject to risks and uncertainties. Please refer to the press release for this event and our SEC filings at intc.com for more information on the risk factors that co
Slide 2: Architecture Day
Architecture Day ol
Slide 3: Architecture Day
Architecture Day Eig intel. 3
Slide 4: Architecture Day
Architecture Day Ea Github Co-Pilot intel. 4
Slide 5: 1000x by 2025
1000x by 2025 1x 5
Slide 6: 1000x by 2025
1000x by 2025 1x 6
Slide 7: 1000x = (Moore's Law) by 2025
5 1000x = (Moore’s Law) by 2025 1x 7
Slide 8: 1000x by 2025: Process & Packaging
1000x by 2025 Process & Packaging 1x 8
Slide 9: 1000x by 2025: Interconnect
1000x by 2025 Interconnect Process & Packaging 1x 9
Slide 10: 1000x by 2025: Memory
1000x by 2025 Memory Interconnect Process & Packaging 1x 10
Slide 11: 1000x: Architecture and Software
1000x Architecture Software Memory Interconnect Process & Packaging 1x 11
Slide 12: Architecture Day
Architecture Day Eig intel. 12
Slide 13: Scalar
Scalar a Architecture Day Bg . intel. 13
Slide 14: Scalar Vector
Scalar Vector Architecture Day Bg intel. 14
Slide 15: Scalar Vector Matrix
Scalar Vector Matrix Architecture Day Bg intel. 15
Slide 16: Scalar Vector Matrix Spatial
Scalar Vector Matrix Spatial 16
Slide 17: Scalar Vector Matrix Spatial
Scalar Vector Matrix Spatial 17
Slide 18: Hybrid Computing Architectures
Hybrid Computing Architectures 18
Slide 19: Hybrid Computing Architectures
Hybrid Computing Architectures Process Packaging Caches Memory Interconnect 19
Slide 20: Hybrid Compute Cluster in a Package
Hybrid Compute Cluster in a Package 20
Slide 21: New Architectural Foundations
New Architectural Foundations Intel Xe - core Performance Efficient Thread Core Director Xe HPC & Sapphire Core Ponte Xe SS Rapids Vecchio Alder AMX Lake Xe HPG Mount Evans
Slide 22: New Architectural Foundations
New Architectural Foundations Intel Xe - core Performance Efficient Thread Core Director Xe HPC & Sapphire Core Ponte Xe SS Rapids Vecchio Alder AMX Lake Xe HPG Mount Evans
Slide 23: Stephen Robinson: Efficient x86 Core
aa fficient x86 Core
Slide 24: Intel's Most Efficient CPU
Intel’s Most Efficient Dense & Highly Performant CPU Scalable Highly Scalable Architecture To Address the Throughput Efficiency Needs For the Next Decade of Compute Vector and AI Instruction Wide Dynamic Support Range 24
Slide 25: Efficient x86 Core Microarchitecture
Intel’s New Efficient x86 Core Microarchitecture Designed for throughput, enabling scalable multi-threaded performance for modern multi-tasking Optimized for power and density efficient throughput with: Deep Wide Optimized Front-End Back-End Design with on-demand with many for latest transistor leng
Slide 26: Instruction Control
Instruction Control Large Instruction Accurate branch Cache (64KB) prediction with an on-demand instruction through deep branch history length decoder accelerates modern and large structure sizes workloads with large code footprints 26
Slide 27: Instruction Control
Instruction Control Dual three wide out of order decoders enable up to 6 instructions per cycle while keeping power and latency in check 27
Slide 28: Data Execution
Data Execution Five-wide allocation with eight-wide retire 256 entry out of order window Discovers data parallelism Seventeen execution ports Executes data parallelism 28
Slide 29: Data Execution
2 FP/Vec Data Execution Store Data 2 Jump ports 4 Integer ALUs 2 FP/Vec stacks 3rd Vec ALU 2 Integer Store Data 2 Load AGUs 2 Store AGUs 29
Slide 30: Memory Subsystem
Memory Subsystem Dual Load + Dual Store Up to 4MB L2 shared among four cores with 64 Bytes/cycle bandwidth in 17 cycles of latency Deep buffering supporting 64 outstanding misses Advanced Prefetchers at all cache levels to detect a wide variety of streams Intel® Resource Director Technology enables
Slide 31: Modern Instruction Set
Modern Instruction Set Support for Security Advanced Vector Instructions with AI extensions Intel® Control-flow Enforcement Technology Wide Vector designed to improve defense in Instruction Set Architecture depth Intel® VT-rp (Virtualization Technology Floating point multiply-accumulate redirect pro
Slide 32: Efficiency in Power and Performance per Transistor
Efficiency in Both Power and Performance per Transistor Intense focus on feature Low switching energy Reduced operating selection and design per instruction voltage required for all implementation costs frequencies to maximize power constrained to maximize area efficiency, which throughput, key for
Slide 33: Latency Performance
Latency Performance Single Thread <40% Power >40% Perf >40% Performance at ISO Power Performance Skylake core 1C1T <40% Power at ISO performance Efficient core 1C1T Power SPECrate2017_int_base estimates using an open source compiler, iso-binary. For workloads and configurations visit www.intel.com/A
Slide 34: Throughput Performance
Throughput Performance Multi Thread +80% Perf +80% More Performance Performance -80% Power Skylake core 2C4T -80% Power at ISO performance Efficient core 4C4T Power SPECrate2017_int_base estimates using an open source compiler, iso-binary For workloads and configurations visit www.intel.com/ArchDay2
Slide 35: Efficient x86 Core Microarchitecture
Intel’s New Efficient x86 Core Microarchitecture Designed for throughput, enabling scalable multi-threaded performance for modern multi-tasking Optimized for power and density efficient throughput with: Deep Wide Optimized Front-End Back-End Design with on-demand with many for latest transistor leng
Slide 36: New Architectural Foundations
New Architectural Foundations Intel Xe - core Performance Efficient Core Thread Deeper, Wider, Optimized Director Xe HPC & Sapphire Core Ponte Xe SS Rapids Vecchio Alder AMX Lake Xe HPG Mount Evans
Slide 37: Adi Yoaz: Performance x86 Core
Performance x86 Core Adi Yoaz
Slide 38: Architecture Goals
Deliver a step function in general purpose CPU performance Architecture Goals Advance the Arch/uArch with new features for evolving trends of workload patterns Innovate with next disruption in AI performance acceleration 38
Slide 39: Performance Core: Wider, Deeper, Smarter
Intel’s New Architecture Designed for speed, pushing the limits of low latency and single threaded application performance via: Wider Deeper Smarter Acceleration of workloads with large code footprint & large data sets NEW AI acceleration technology via coprocessor for matrix multiplication NE
Slide 40: Front-End
Front-End Smarter Fetch instructions and decodes them into µops Improved branch prediction accuracy Smarter code prefetch mechanism Large Code 128256 4K iTLB, 1632 2M/4M iTLB Enhanced code prefetch 5K12K branch targets Wider 16B32B length decode μop$ 46 decoders 2.25K4K μops: 68
Slide 41: Out of Order Engine
Out of Order Engine Track μop dependencies and dispatch ready μops to execution units Wider 5 6 wide allocation 10 12 execution ports Deeper 512-entry Reorder-Buffer and larger Scheduler sizes Smarter More instructions “executed” at rename / allocation stage 41
Slide 42: Integer Execution Units
Integer Execution Units 5th Integer execution port / ALU added 1-cycle LEA on all 5 ports Used also for arithmetic calculations 42
Slide 43: Vector Execution Units
Vector Execution Units New Fast Adder (FADD): Power efficient, low latency FMA units support FP16 data type FP16 added to Intel® AVX512 including complex numbers support 43
Slide 44: L1 Cache & Memory Subsystem
L1 Cache & Memory Subsystem Wider Smarter 2 3 load ports: Reduced effective Load Latency 3×256bit loads Faster Memory Disambiguation 2×512bit loads resolution Large Data Deeper DTLB 64 96 Deeper Load Buffer and Store Buffer L1 D$: 12 16 fill buffers expose more memory parallelism L1D
Slide 45: L2 Cache & Memory Subsystem
L2 Cache & Memory Subsystem Bigger L2$: 1.25MB (client) or 2MB (data center) Faster Max demand misses 3248 Smarter L2$ pattern-based multi-path prefetcher Feedback-based prefetch throttling Full-line-write predictive bandwidth optimization – reduces DRAM reads 45
Slide 46: ISO Frequency Performance vs. Cypress Cove Core
General-Purpose Performance Vs. 11th Gen Intel® Core™ 19% 1.6x 1.5x ISO Frequency Performance vs. Cypress Cove Core Performance Improvement at 1.4x ISO Frequency1 1.3x 1.2x 1.1x 1.0x 0.9x 0.8x 0.7x 0.6x 0.5x 0.4x 1 Geomean of Performance core (ADL) vs. Cypress Cove (RKL) Core @ ISO 3.3GHz Frequency
Slide 47: Intel Advanced Matrix Extensions (AMX)
Intel® Advanced Matrix Extensions (Intel® AMX) Tiled Matrix Multiplication Accelerator - Data Center AMX 2048 int8 8x operations / cycle / core VNNI 256 int8 47
Slide 48: Intel AMX: Tiles and TMUL
Intel® Advanced Matrix Extensions (Intel® AMX) Tiled Matrix Multiplication Accelerator - Data Center AMX architecture has two components: Tiles • A new expandable 2D register file – 8 new registers, 1Kb each: T0-T7 • Register file supports basic data operators – load/store, clear, set to constant, etc
Slide 49: Intel AMX Architecture
Intel® Advanced Matrix Extensions (Intel® AMX) Architecture TMUL Engine TILES and Intel Architecture Coprocessor 1 coprocessor Host commands Tmm0 += Tmm1 * Tmm2 TILECONFIG Coherent tmm0 Memory Coprocessor 2 Interface tmm1 tmm[n-1] 49
Slide 50: A Step Function in CPU Architecture
New A Step Function in CPU Architecture Performance For the Next Decade of Compute A significant IPC boost at high power efficiency Wider Deeper Smarter Better supports large data set and large code footprint applications Enhanced power management improves frequency and power Machine Learning Techno
Slide 51: New Architectural Foundations
New Architectural Foundations Intel Xe - core Efficient Core Thread Deeper, Wider, Optimized Director Xe HPC & Performance Sapphire Ponte Core Biggest Shift in x86 yet Xe SS Rapids Vecchio AMX Advanced Matrix Alder Extension - Engine Lake Xe HPG Mount Evans
Slide 52: Scalar Architecture Roadmap
What we said Architecture Scalar Architecture Roadmap Day 2020 Coves Monts 2019 Today 2021 52
Slide 53: Graph is for conceptual illustration purposes only
Graph is for conceptual illustration purposes only. 53
Slide 54: Rajshree Chabukswar: Intel Thread Director
-ety 46 Intel Thread Director | t p U-,! CU
Slide 55: Performance Hybrid Scheduling Goals
Performance Hybrid Scheduling Goals Software Transparent Real -Time Adaptive Scalable from Mobile to Desktop 55
Slide 56: Introducing Intel Thread Director
Introducing OS Scheduler Intel Thread Director Intelligence built directly into the core Monitors the runtime instruction mix of each thread and as well as the state of each core – with nanosecond precision Provides runtime feedback to the OS to make the optimal scheduling decision for any workload
Slide 57: Intel Thread Director Scheduling Examples
Introducing Intel Thread Director P Scheduling Examples 57
Slide 58: Intel Thread Director Scheduling Examples
Introducing Intel Thread Director P Scheduling Examples 1 Priority tasks scheduled on P-cores 58
Slide 59: Intel Thread Director Scheduling Examples
Introducing B Intel Thread Director B B Scheduling Examples 1 Priority tasks scheduled on P-cores 2 Background tasks scheduled on E-cores 59
Slide 60: Intel Thread Director Scheduling Examples
Introducing Intel Thread Director AI Scheduling Examples 1 Priority tasks scheduled on P-cores 2 Background tasks scheduled on E-cores 3 New AI thread ready 60
Slide 61: Intel Thread Director Scheduling Examples
Introducing Intel Thread Director AI Scheduling Examples 1 Priority tasks scheduled on P-cores 2 Background tasks scheduled on E-cores 3 New AI thread ready 61
Slide 62: Intel Thread Director Scheduling Examples
Introducing Intel Thread Director AI Scheduling Examples 1 Priority tasks scheduled on P-cores 2 Background tasks scheduled on E-cores 3 AI thread prioritized on P-core 62
Slide 63: Intel Thread Director Scheduling Examples
Introducing Intel Thread Director Scheduling Examples 1 Priority tasks scheduled on P-cores 2 Background tasks scheduled on E-cores 3 AI thread prioritized on P-core 4 Spin loop wait moved from P to E-core 63
Slide 64: Intel Thread Director Scheduling Examples
Introducing Intel Thread Director Scheduling Examples 1 Priority tasks scheduled on P-cores 2 Background tasks scheduled on E-cores 3 AI thread prioritized on P-core 4 Spin loop wait moved from P to E-core 64
Slide 65: Intel Thread Director Scheduling Examples
Introducing Intel Thread Director Scheduling Examples 1 Priority tasks scheduled on P-cores 2 Background tasks scheduled on E-cores 3 AI thread prioritized on P-core 4 Spin loop wait moved from P to E-core 65
Slide 66: New Architectural Foundations
New Architectural Foundations Intel Thread Director Scalable Hybrid Arch. Scheduling Xe - core Efficient Core Deeper, Wider, Optimized Xe HPC & Performance Sapphire Ponte Core Biggest Shift in x86 yet Xe SS Rapids Vecchio AMX Advanced Matrix Alder Extension - Engine Lake Xe HPG Mount Evans
Slide 67: Arik Gihon
Arik Gihon
Slide 68: Introducing Alder Lake
Introducing Alder Lake Reinventing Multi Core Architecture Single, Scalable SoC Architecture All Client Segments – 9W to 125W – built on Intel 7 process All-New Core Design Performance Hybrid with Intel Thread Director Industry-Leading Memory & I/O DDR5, PCIe Gen5, Thunderbolt™ 4, Wi-Fi 6E 68
Slide 69: Scalable Client Architecture
Scalable Client Architecture Desktop Mobile Ultra Mobile LGA 1700 BGA Type3 BGA Type4 HDI Socket 50 x 25 x 1.3 mm 28.5 x 19 x 1.1 mm Visit www.intel.com/ArchDay21claims for details 69
Slide 70: Alder Lake Building Blocks
Alder Lake Building Blocks Xe Architecture P-Core E-Cores IPU GNA 3.0 SOC Memory LLC Display PCIe TBT 70
Slide 71: Building Blocks
Desktop Mobile Ultra Mobile Building Blocks Memory LLC SOC Display PCIe TBT GNA 3.0 IPU P-Core E-Cores 71
Slide 72: Building Blocks
Desktop Mobile Ultra Mobile GNA Display PCIe 3.0 LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC Memory Building Blocks Memory LLC SOC Display PCIe TBT GNA 3.0 IPU P-Core E-Cores 72
Slide 73: Building Blocks
Desktop Mobile Ultra Mobile GNA GNA Display PCIe PCIe TBT TBT 3.0 3.0 Display IPU TBT TBT LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC Memory Memory Building Blocks Memory LLC SOC Display PCIe TBT GNA 3.0 IPU P-Core E-Cores 73
Slide 74: Building Blocks
Desktop Mobile Ultra Mobile GNA GNA Display PCIe PCIe TBT TBT 3.0 3.0 Display IPU TBT TBT LLC LLC GNA PCIe LLC LLC 3.0 TBT TBT LLC LLC Display IPU LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC LLC Memory Memory Memory Building Blocks Memory LLC SOC Display PCIe TBT GNA 3.0 IPU P-Core E
Slide 75: Alder Lake Core/Cache
Alder Lake Core/Cache Up To Up To Up to 16 Cores 24 Threads 30MB 8 Performance 2T per P-core Non-inclusive 8 Efficient 1T per E-core LL Cache 75
Slide 76: Alder Lake Memory Subsystem
Alder Lake Memory Memory Subsystem DDR5 – 4800 New Leading the industry DDR4 - 3200 transition to DDR5 DDR PHY LP5 - 5200 New Support for all four major memory technologies LP4x - 4266 Dynamic voltage-frequency scaling Enhanced overclocking support 76
Slide 77: Alder Lake PCIe
Alder Lake PCIe x16 PCIe Gen 5 New x4 PCIe Gen 4 PCIe Leading the industry transition to PCIe Gen5 Up to 2X bandwidth vs. Gen4 Up to 64GB/s with x16 lanes x12 PCIe Gen 4 PCIe x16 PCIe Gen 3 PCIe Visit www.intel.com/ArchDay21claims for details 77
Slide 78: Alder Lake Interconnect
I/O Fabric Alder Lake Memory Subsystem Up to Interconnect 64 GB/s Real-time, demand- Compute Fabric based BW control Copmpute Fabric Up to 1000 GB/s Memory Fabric Dynamic Latency Up To Optimization 204 GB/s Dynamic Bus Width & Frequency Visit www.intel.com/ArchDay21claims for details 78
Slide 79: Alder Lake: Reinventing Multi Core Architecture
Beginning Fall 2021 Alder Lake Reinventing Multi Core Architecture Single, Scalable SoC Architecture All Client Segments – 9W to 125W – built on Intel 7 process All-New Core Design Performance Hybrid with Intel Thread Director Industry-Leading Memory & I/O DDR5, PCIe Gen5, Thunderbolt™ 4, Wi-Fi 6E 7
Slide 80: New Architectural Foundations
New Architectural Foundations Intel Thread Director Scalable Hybrid Arch. Scheduling Xe - core Efficient Core Deeper, Wider, Optimized Xe HPC & Performance Sapphire Ponte Core Biggest Shift in x86 yet Xe SS Rapids Vecchio Alder Lake Performance Hybrid AMX Advanced Matrix Extension - Engine Xe HPG Mo
Slide 81: Xe HPG architecture
xe HPG architecture >
Slide 82: Leadership Integrated Graphics Performance
Leadership Integrated Graphics Performance 2x Gen 11 2x Gen 9 Power For workloads and configurations visit www.intel.com/ArchDay21claims. Results may vary. 82
Slide 83: Unconstrained Discrete Graphics Performance
Unconstrained Discrete Graphics Performance 2x Gen 11 2x Gen 9 Power For workloads and configurations visit www.intel.com/ArchDay21claims. Results may vary. 83
Slide 84: Vivid PC Graphics Market
Vivid PC Graphics Market 1.5B 8.8B 13M+ PC Gamers Hours of Game Live Streams Developers Over the last 4 years, the amount of concurrent users Watched has doubled on Steam. Over 10,000 games Twitch.tv viewership has released on Steam in 2020 doubled in one year 1. Source: https://www.pcgamesn.com/pc-
Slide 85: Powered by Architecture Day
intel Powered by Architecture Day 222 intel. 85
Slide 86: Xe HPG Sneak Peek: Lisa Pearce
xe HPG Sneak Peek Lisa Pearce
Slide 87: Software First
Software First APIs and Engines Core Driver Experiences DDI Smooth Gaming Intel Graphics Memory DDI Compiler Management Threading Live Game Streaming GPU Profiler Submission Command Modern User Interface Encoding Performance Tuning OS/GPU 87
Slide 88: Performance vs Quality Trade-off
Low Quality Image Render Quality Performance Trade-off High Quality Image Quality 88
Slide 89: Traditional Upscaling
Low Quality Image Traditional Upscaling Render Quality Performance Trade-off Traditional Upscaling High Quality Image Quality 89
Slide 90: Performance vs Quality Trade-off
Low Quality Image Render Quality Performance Trade-off High Quality Image Quality 90
Slide 91: Performance vs Quality Trade-off
Low Quality Image Render Quality Performance Trade-off High Quality Image Quality 91
Slide 92: Xe Super Sampling Pipeline
Velocity History Xe Super Sampling Raster & Lighting Post Process Jitter Frame N 92
Slide 93: Xe SS
High Quality 4K Image Xe SS 4K Xe SS + XMX Hits the Xe SS + DP4a 4K Sweet Spot Traditional Upscaling “4K” Low Quality 1080p Image Frame Render Time Graph is for conceptual illustration purposes only. Subject to revision with further testing. 93
Slide 94: Xe SS SDK Available this month
X SS e SDK Available this month 94
Slide 95: Xe HPG Sneak Peek: David Blythe
xe HPG Sneak Peek David Blythe
Slide 96: High Performance Gaming Optimized
High Performance Scalability Gaming Optimized 96
Slide 97: Compute Building Block of Xe HPG-based GPUs
Compute Building Block of Xe HPG-based GPUs 16 16 Vector Engines Matrix Engines 256 bit 1024 bit per engine per engine 97
Slide 98: Render Slice
7 XcHPG Render Slice Architecture Day Eig Render slice a | = 6 usisin | ‘Sampler Rasterizer Pixel Backend Pixel Backend intel. 98
Slide 99: Xe-cores with XMX
4 X°-cores with XMX Architecture Day Eig YR core intel. 99
Slide 100: Fixed Function optimized for DX12 Ultimate Gaming
Fixed Function optimized for DX12 Ultimate Gaming Geometry Pipeline Rasterization Pipeline Samplers Pixel Backends 100
Slide 101: Ray Tracing Units
4 Ray Tracing Units Ray Traversal Bounding Box Intersection Triangle Intersection 101
Slide 102: Architecture Day
Architecture Day Eig Vector Vector ‘Sampler | Pixel Backend Geometry Rasterizer Pixel Backend intel. 102
Slide 103: Scaling the Graphics Engine
Scaling the Graphics Engine 103
Slide 104: Scaling the Graphics Engine
Scaling the Graphics Engine 104
Slide 105: Leadership IP Performance/Watt
Leadership IP Performance/Watt Xe HPG Architecture Xe HPG Logic Design 1.5x Performance/Watt Frequency 1.5x Xe LP Circuit Design Xe LP Process Technology Discrete Software Voltage For workloads and configurations visit www.intel.com/ArchDay21claims. Results may vary. 105
Slide 106: TSMC N6 quote from Dr. Kevin Zhang
“In the world of graphics, there is an insatiable demand for better performance and more realism. TSMC is excited that Intel has chosen our N6 technology for their Alchemist family of discrete graphics solutions”. “There are many ingredients to a successful graphics product including the semiconduct
Slide 107: Multi-Year Xe Architecture Roadmap
Druid Multi-Year Xe Next Architecture Celestial Xe3 HPG Performance Battlemage Xe2 HPG Xe HPG Q1 2022 107
Slide 108: New Architectural Foundations
New Architectural Foundations Intel Thread Xe - core Director Foundational Scalable Hybrid Building Block for Xe Arch. Scheduling With Xe Matrix Extensions Efficient Core Deeper, Wider, Optimized Xe HPC & Performance Sapphire Ponte Core Biggest Shift in x86 yet Xe SS Rapids Vecchio Alder Lake Perfor
Slide 109: Part Recap: New Architectural Foundations
Part 1 Recap Intel Thread Xe - core Director Foundational Scalable Hybrid Building Block for Xe Arch. Scheduling With Xe Matrix Extensions Efficient Core Deeper, Wider, Optimized Xe HPC & Performance Sapphire Ponte Core Biggest Shift in x86 yet Xe SS Rapids Vecchio Alder Lake Performance Hybrid AMX
Slide 110: Sailesh Kottapalli: Sapphire Rapids
a ds Sapphire Rap i. } mo 5a
Slide 111: Introducing Sapphire Rapids
New Standard for Data Center Architecture Introducing Designed for Microservices & AI Workloads Next-Gen Intel Xeon Scalable Processor Pioneering Advanced Memory & IO Transitions 111
Slide 112: Node and Data Center Performance
Node Performance Data Center Performance 112
Slide 113: Node Performance
Node Performance Cache & Scalar Data Parallel Memory Sub- Intra/Inter Performance Performance System Arch Socket Scaling Larger Private & Modular SoC /w New Performance Multiple Integrated Shared Caches Modular Die Fabric Core Acceleration Microarchitecture Engines DDR 5 Wider & Faster UPI Increased
Slide 114: Data Center Performance
Next Gen Quality of Service Capabilities Broad WL/Usage Support and Low Jitter Optimizations Fast VM Migration Architecture Consistent Caching Next Gen Optane Integrated WL Better Telemetry & Mem Latency Support Accelerators Inter-Processor Improved Security & IO Virtualization CXL 1.1 Interrupt Vir
Slide 115: Multi-Tile Design for Scalability
Single Monolithic Die Multi-Tile Design for Increased Scalability Delivers a scalable, balanced architecture leveraging existing software paradigms for monolithic CPUs via a modular architecture 115
Slide 116: Multiple Tiles, Single CPU
Sapphire Rapids Multiple Tiles, Single CPU Every thread has full access to all resources on all tiles Cache, Memory, IO… Provides consistent low latency & high cross-section BW across the entire SoC 116
Slide 117: Sapphire Rapids SoC
Sapphire Rapids SoC 117
Slide 118: Sapphire Rapids Key Building Blocks
Sapphire Rapids Key Building Blocks Compute IP Cores Acceleration Seamless Integration of Engines PCIe I/O IP CXL1.1 Gen 5 UPI 2.0 Memory DDR5 Optane HBM IP 118
Slide 119: Performance Core Built for Data Center
Performance Core Built for Data Center Major microarchitecture and IPC improvement Improved support for large code/data footprint Consistent performance for multi-tenant usages Autonomous/Fast PM for high freq @ low jitter 119
Slide 120: Performance Core Architecture Improvements
AI Intel® Advanced Matrix Extensions - AMX Tiled matrix operations for inference & training acceleration Performance Core Attached Device Accelerator interfacing Architecture - AiA Efficient dispatch, signaling & synchronization from user level Architecture Improvements for DC Half- Precision Worklo
Slide 121: Acceleration Engines
Sapphire Rapids Utilization Without Acceleration Acceleration Engines Increasing effectiveness of cores, by enabling offload of common mode tasks via seamlessly integrated acceleration engines Core Core Core Core With Acceleration Native Dispatch, Signaling & Synchronization from User Space Accelera
Slide 122: Intel Data Streaming Acceleration Engine
Intel® Data Streaming Acceleration Engine Optimizing streaming data movement and transformation operations up to 4 Instances per Socket 14% 39% Core utilization of Open vSwitch 53% Low Latency Invocation @ 1512B packet size 47% 86% additional CPU Core cycles after No Memory Pinning Overhead DSA offl
Slide 123: Intel Quick Assist Technology Acceleration Engine
Intel® Quick Assist Technology Acceleration Engine Accelerating Cryptography and Data De/Compression 100% up to Expected Core utilization on SPR for Zlib L9 400Gb/s Symmetric Crypto up to 160Gb/s Compression + 160Gb/s De-compression 98% additional workload capacity Fused Operations after QAT offload
Slide 124: Sapphire Rapids SoC
Sapphire Rapids SoC 124
Slide 125: Sapphire Rapids I/O Advancements
Sapphire Rapids I/O Advancements Introducing Compute eXpress Link (CXL) 1.1 Accelerator and memory expansion in datacenter Expanded device performance via PCIe 5.0 & connectivity Improved DDIO & QoS capabilities Improved Multi-Socket scaling via Intel® Ultra Path Interconnect (UPI) 2.0 Up to 4 x24 U
Slide 126: Memory and Last Level Cache
Sapphire Rapids Memory and Last Level Cache Increased Shared Last Level Cache (LLC) Up to >100 MB LLC shared across ALL cores Increased bandwidth, security & reliability via DDR 5 Memory 4 memory controllers supporting 8 channels Intel® Optane™ Persistent Memory 300 Series 126
Slide 127: High Bandwidth Memory
Sapphire Rapids High Bandwidth Memory Significantly Higher Memory Bandwidth vs. baseline Xeon-SP with 8 channels of DDR 5 Increased capacity and Bandwidth some usages can eliminate need for DDR entirely 2 Modes HBM Flat Mode HBM Caching Mode Flat Mem Regions w/ HBM & DRAM DRAM backed cache 127
Slide 128: Sapphire Rapids - Architected for AI
Sapphire Rapids - Architected for AI AI has become ubiquitous across usages – AI performance required in all tiers of computing Goal Enable efficient usage of AI across all services deployed on 2048 elastic general-purpose tier by delivering many times more AI performance and lower CPU utilization O
Slide 129: Sapphire Rapids - Built for Microservices
Sapphire Rapids - Built for elastic computing models - microservices >80% of new cloud-native and SaaS applications are expected to be built as microservices Goal Enable higher throughput while meeting latency requirements and Microservices Performance reducing infrastructure overhead for execution,
Slide 130: New Standard in Data Center Architecture
New Standard in Data Center Architecture General Purpose Sapphire Rapids Multi Tile SoC for Physically Tiled, & Dedicated Biggest Leap in Data Center Capabilities Scalability Logically Monolithic Acceleration Engines in over a Decade Designed for Microservices and AI Workloads Performance Core Workl
Slide 131: New Architectural Foundations
New Architectural Foundations Intel Thread Xe - core Director Foundational Scalable Hybrid Building Block for Xe Arch. Scheduling With Xe Matrix Extensions Efficient Core Sapphire Deeper, Wider, Optimized Rapids Performance Biggest Xe HPC & Core Leap in DC Capabilities Ponte Biggest Shift in x86 yet X
Slide 132: Guido Appenzeller
Guido Appenzeller
Slide 133: Server Architecture in a Classic Data Center
Server Architecture in a classic Data Center Software and Infrastructure are all controlled by One Entity Server Hardware General Purpose Compute Network Interface Card Virtualized Network CPU NIC Everything runs on the CPU 133
Slide 134: Classic vs Cloud Server Architecture
Classic Server Architecture General Purpose Compute Network Interface Card Virtualized Network Server Hardware NIC CPU Everything runs on the CPU Cloud Server Architecture General Purpose Compute Infrastructure Processing Unit Virtualized Network Server Hardware IPU IPU CPU Tenant Code & CSP Code Hy
Slide 135: Rooms and Kitchen analogy
Rooms Guest Kitchen Dinner Table Kitchen Living Room Dining Hall 135
Slide 136: Major Advantages of IPUs
Major Advantages of IPUs 1 2 3 CPU CPU IPU IPU Separation of Infrastructure Infrastructure Diskless Server & Tenant Offload Architecture Guest can fully control the CPU Accelerators help process Simplifies data center with their SW, while CSP these task efficiently. Minimize architecture while addin
Slide 137: Separation of Infrastructure and Tenant
Advantage 1 - Separation of Infrastructure and Tenant Maximum Control and Isolation for the Tenant Management Software Network Storage Customer’s Software Hardware Virtualized Network CPU CPU IPU IPU 137
Slide 138: Infrastructure Offload
Advantage 2 - Infrastructure Offload In some cases, the majority of CPU cycles are spent on overhead % Cycles spent in microservice functionalities 100% Microservice 50% Overhead at Facebook 0% Web Feed 1 Feed 2 Ads 1 Ads 2 Cache 1 Cache 2 Source: From Accelerometer: Understanding Acceleration Opportu
Slide 139: Infrastructure Offload
Advantage 2 - Infrastructure Offload Dedicated Accelerators Free up CPU Capacity Management Software Software Based Network Storage Customer’s Software Accelerators for Efficient Processing Standard OS Hardware CPU CPU IPU IPU 139
Slide 140: Diskless Server Architecture
Advantage 3 - Diskless Server Architecture Scale with Virtual Storage via Network Control Pane Create New Virtual SSD Storage Service Management Software Customer’s Network Storage Software Create New Virtual Storage data path Volume - SSD1 Standard OS is fully implemented in hardware! NVMe – SSD1 C
Slide 141: Broad Infrastructure Acceleration Portfolio
Broad Infrastructure Acceleration Portfolio Dedicated ASIC IPU FPGA-based Acceleration IPU Platforms & Adapters SmartNICs Faster time to market for evolving standards Performance and power optimized Re-programmable Secure Datapath enables Programmable accelerated flexible/customizable workload offlo
Slide 142: Introducing Oak Springs Canyon
Introducing Oak Springs Canyon High perf networking and storage acceleration for Cloud Service Providers OVS, NVMe over Fabric, and RoCE solutions Programmable through Intel OFS, DPDK, and SPDK Customizable solutions with FPGA 142
Slide 143: Oak Springs Canyon
Oak Springs Canyon Built with Intel® Agilex FPGA and Xeon-D SoC DDR4 DDR4 16GB 16GB High speed Ethernet support - 2x100G PCIe Gen 4 x16 PCIe Hardware crypto block enables Gen 4 x16 security at line rate 143
Slide 144: Introducing Arrow Creek
Introducing Arrow Creek Acceleration Development Platform (ADP) for High Performance 100G networking acceleration Customizable packet processing including bridging and networking services Programmable through Intel OFS and DPDK Accelerated infrastructure workloads Juniper Contrail , OVS, SRv6, vFW S
Slide 145: Arrow Creek
Arrow Creek Built with Intel® Agilex FPGA and Ethernet E810 Controller 2x QSFP56 supporting DDR4 up to 2x100GbE networking 1GB PCIe Gen4 x16 DDR4 4GB 4GB 16GB DDR4 memory 1GB DDR4 to integrated CPU Precision 4GB Clocking 4GB Full height, half length PCIe form PCIe PCIe factor; passively cooled Gen 4
Slide 146: Naru Sundar
Naru Sundar
Slide 147: Introducing Mount Evans
Introducing Mount Evans Hyperscale Co-designed with a top cloud provider Integrated learnings from multiple gen. of FPGA sNICs Intel’s 200G IPU Ready High performance under real world load Security and isolation from the ground up Best-in-Class Programmable Packet Processing Engine Technology NVMe s
Slide 148: Mount Evans Architectural Breakdown
Mount Evans Architectural Breakdown Network Compute Subsystem Complex 148
Slide 149: Mount Evans Network Subsystem
Mount Evans Architectural Breakdown Support for up to 4 host Xeons with 200Gb/s full duplex High-performance ROCEv2 NVMe offload engine Programmable packet pipeline with QoS and telemetry capabilities Inline IPSec 149
Slide 150: Mount Evans Compute Complex
Mount Evans Compute Complex Up to 16 Arm Neoverse® N1 Cores Dedicated compute and cache with up to 3 memory channels Lookaside crypto and compression Dedicated management processor 150
Slide 151: New Architectural Foundations
New Architectural Foundations Intel Thread Xe - core Director Foundational Scalable Hybrid Building Block for Xe Arch. Scheduling With Xe Matrix Extensions Efficient Core Sapphire Deeper, Wider, Optimized Rapids Performance Biggest Xe HPC & Core Leap in DC Capabilities Ponte Biggest Shift in x86 yet
Slide 152: Xe HPC architecture
xe HPC architecture a
Slide 153: HPC FP64 Bandwidth GB/s
HPC FP64 Bandwidth GB/s For workloads and configurations visit www.intel.com/ArchDay21claims . Results may vary. 153
Slide 154: HPC FP64 AI FP16/BF16 Bandwidth GB/s
HPC FP64 AI FP16/BF16 Bandwidth GB/s For workloads and configurations visit www.intel.com/ArchDay21claims . Results may vary. 154
Slide 155: HPC FP64 AI FP16/BF16 Bandwidth GB/s
HPC FP64 AI FP16/BF16 Bandwidth GB/s For workloads and configurations visit www.intel.com/ArchDay21claims . Results may vary. 155
Slide 156: Hong Jiang
4 © Architecture X\ upc Hong Jiang
Slide 157: Compute Building Block of Xe HPC-based GPUs
Compute Building Block of Xe HPC-based GPUs Load / Store 8 8 512 B/CLK Vector Matrix Engines Engines Cache 512 bit 4096 bit per engine per engine L1$/ SLM (512KB), I$ 157
Slide 158: Core
Core Vector Engine Matrix Engine (ops/clk) (ops/clk) 256 FP32 2048 TF32 256 FP64 4096 FP16 512 FP16 4096 BF16 8192 INT8 158
Slide 159: Xe slice
ia d 7@ « XE, slice Wool Aa: Tana aa: a) ee ae) 159 intel. Architecture Day
Slide 160: Slice
Slice 16 Xe – cores 8MB L1 Cache 160
Slide 161: Slice
Slice 16 Xe – cores 8MB L1 Cache 16 Ray Tracing Units Ray Traversal Triangle Intersection Bounding Box Intersect. 161
Slide 162: Slice
Slice 16 Xe – cores 8MB L1 Cache 16 Ray Tracing Units Ray Traversal Triangle Intersection Bounding Box Intersect. 1 Hardware Context 162
Slide 163: Stack
Stack Up to 4 Slices 64 Xe - cores 64 Ray Tracing Units 4 Hardware Contexts L2 Cache 4 HBM2e controllers 1 Media Engine 8 Xe Links 163
Slide 164: 2-Stack
2 - Stack 2 8 Slices Media Engines 128 Xe - cores 8 HBM2e 128 Ray Tracing Units controllers 8 Hardware Contexts 16 X Links e For workloads and configurations visit www.intel.com/ArchDay21claims . Results may vary. 164
Slide 165: Xe Link
Link High Speed Coherent Unified Fabric (GPU to GPU) Load/Store, Bulk Data Transfer & Sync Semantics Up to 8 Fully Connected GPUs through Embedded Switch 165
Slide 166: Xe Link for Scalability
Link for Scalability Xe - Link Xe - Link 166
Slide 167: Xe Link for Scalability
Link for Scalability Xe - Link Xe - Link Xe - Link Xe - Link 167
Slide 168: Xe Link for Scalability
Link for Scalability Xe - Link Xe - Link 168
Slide 169: Xe Link for Scalability
Link for Scalability Xe - Link Xe - Link Xe - Link Xe - Link 169
Slide 170: 8x System Compute Rates
Xe - Link 8x System Compute Rates Vector Matrix Xe - Link Xe - Link Up to Up to 32,768 262,144 FP64 Ops/CLK TF32 Ops/CLK Up to Up to 32,768 524,288 FP32 Ops/CLK BF16 Ops/CLK Xe - Link Up to 1,048,576 INT8 Ops/CLK 170
Slide 171: Ponte Vecchio: Masooma Bhaiwala
Ponte Vecchio Masooma Bhaiwala
Slide 172: Ponte Vecchio
Ponte Vecchio
Slide 173: Ponte Vecchio: New Technologies
New Verification Methodology New Software New Reliability Methodology New Signal Integrity Techniques New Interconnects New Power Delivery Technology New Packaging Technology New I/O Architecture New Memory Architecture New IP Architecture New SOC Architecture 173
Slide 174: Ponte Vecchio
Compute Tile Ponte Rambo Tile Vecchio Foveros SOC Base Tile >100 Billion Transistors HBM Tile 47 Active Tiles Xe Link Tile 5 Process Nodes Multi Tile Package EMIB Tile 174
Slide 175: Ponte Vecchio Key Challenges
Compute Tile Ponte Rambo Tile Vecchio Foveros Key Challenges Base Tile HBM Tile Scale of Integration Xe Link Tile Foveros Implementation Multi Tile Package Verification Tools & Methods EMIB Tile Signal Integrity, Reliability & Power Delivery 175
Slide 176: Ponte Vecchio Compute Tile
Ponte Compute Tile Vecchio Compute Tiles Per Tile L1 Cache 8 4MB Xe - cores Per Tile Built on Bump Pitch TSMC 36um N5 Foveros 176
Slide 177: Ponte Vecchio Base Tile
Ponte Vecchio Base Tile Base Tile Built on Area HBM2e Intel 7 640mm2 FOVEROS MDFI L2 Cache Host Interface 144MB PCIe Gen5 EMIB 177
Slide 178: Ponte Vecchio Xe Link Tile
Ponte Vecchio Xe Link Tile Xe Link Tile 8 ports Per Tile Embedded 8 Xe Links Switch Up to Built on TSMC N7 90G Serdes For workloads and configurations visit www.intel.com/ArchDay21claims . Results may vary. 178
Slide 179: Ponte Vecchio A0 Silicon Current Status
Ponte A0 Silicon Current Status Vecchio FP32 Throughput Execution Progress Memory Fabric Bandwidth Connectivity Bandwidth For workloads and configurations visit www.intel.com/ArchDay21claims . Results may vary. 179
Slide 180: Accelerated Compute Systems
Accelerated Compute Systems Ponte Vecchio x4 Subsystem with Xe Links + 2S Sapphire Rapids Ponte Vecchio x4 Subsystem with Xe Links Ponte Vecchio OAM 180
Slide 181: Overcoming Separate CPU and GPU Software Stacks
Overcoming Separate CPU and GPU Software Stacks CPU-Optimized Stack GPU-Optimized Stack Applications & Services Applications & Services Middleware, Frameworks & Runtimes Middleware, Frameworks & Runtimes Low-level Libraries Languages Low-level Libraries Languages Hardware Abstraction Layer Hardware
Slide 182: Open, Standards-Based Unified Software Stack
CPU & XPU - Optimized Stack Applications & Services Middleware, Frameworks & Runtimes ... Low-level Libraries Languages Open, Standards-Based oneMKL oneDNN oneDAL oneVPL DPC++ Unified Software Stack Other oneTBB oneCCL oneDPL Libraries Other Languages Freedom from proprietary programming models Full
Slide 183: oneAPI Industry Momentum
oneAPI Industry Momentum Cross-Vendor Evolving Spec Nvidia GPU Arm CPU + Graph interfaces for Deep Learning workloads Huawei ASIC AMD GPU + Advanced raytracing libraries 183
Slide 184: Industry Momentum
Industry Momentum End Users National Labs ISVs & OSVs OEMs & SIs Universities & Research Institutes CSPs & Frameworks Indian Institutes of Technology Delhi / Kharagpur / Roorkee Indian Institute of Indian Institute of Science Science Education UNIVERSITY OF University Bangalore & Research Pune CAMBR
Slide 185: oneAPI developers and applications
>200K Developers >300 Applications >80 HPC & AI Applications Unique installs of Intel® oneAPI product Deployed in market using Intel® oneAPI Functional on Intel’s Xe HPC architecture since Dec’20 release language & libraries using Intel® oneAPI Toolkits v2021.3 Available Now 185
Slide 186: oneAPI developers and applications
>200K Developers >300 Applications >80 HPC & AI Applications Unique installs of Intel® oneAPI product Deployed in market using Intel® oneAPI Functional on Intel’s Xe HPC architecture since Dec’20 release language & libraries using Intel® oneAPI 186
Slide 187: oneAPI developers and applications
>200K Developers >300 Applications >80 HPC & AI Applications Unique installs of Intel® oneAPI product Deployed in market using Intel® oneAPI Functional on Intel’s Xe HPC architecture since Dec’20 release language & libraries using Intel® oneAPI 187
Slide 188: oneAPI developers and applications
>200K Developers >300 Applications >80 HPC & AI Applications Unique installs of Intel® oneAPI product Deployed in market using Intel® oneAPI Functional on Intel’s Xe HPC architecture since Dec’20 release language & libraries using Intel® oneAPI 188
Slide 189: Blade: Building Block for the ExaScale Supercomputer
Blade Building Block for the ExaScale Supercomputer 189
Slide 190: Leadership Performance for HPC/AI
Leadership Performance for HPC/AI Connectivity to drive scaleup The vision 2 years ago… and scale out Unified Programming Model powered with oneAPI 190
Slide 191: Architecture Day
Architecture Day ol
Slide 192: New Architectural Foundations
New Architectural Foundations Intel Thread Xe - core Xe HPC & Director Foundational Ponte Vecchio Scalable Hybrid Building Block for Xe With Xe Matrix Extensions Back to FLOP Arch. Scheduling Efficient Leadership Core Sapphire Deeper, Wider, Optimized Rapids Biggest Performance Leap in DC Core Capab
Slide 193: See you at Intel Innovation
See you at intel. INNnovatION ure Day intel. 193
Slide 194: Architecture Day
Architecture Day ol
Slide 195: Notices & Disclaimers
Notices & Disclaimers Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex.Performance results are based on testing as of dates shown in configurations and may not reflect all publicly available updates. See www.Intel.com/ArchDay21claims for config