Keysight Technologies Inc. · NYSE:KEYS
Keysight Technologies — DesignCon 2025
18 slides from Keysight Technologies's conference presentation, Feb 27, 2025
Slide 1: Benchmarking Collective Operations: A Key to Optimizing AI Infrastructure Performance
Benchmarking Collective Operations: A Key to Optimizing AI Infrastructure Performance Alex Bortok & Sushil Srinivasan 1
Slide 2: AI Cluster Test Framework
AI Cluster Test Framework Emulate AI Workloads & Network Applications Fabric, Benchmark Network Fabrics Complementary adjacencies aligned to Systems and Optimize Network performance to customers’ expanding design & test needs AI Models meet complex application demands + Servers, Device Interoperabil
Slide 3: Why the Network & Components Matters in an AI Cluster
Why the Network & Components Matters in an AI Cluster AI is Compute, Network & Data Intensive and requires validation at System Scale 100% 90% 80% 70% Network failures 60% >20% 50% 40% Waiting 30% GPUs waiting on data 20% >50% 10% 0% GPU Time Memory 20% failures Computation Source: Unicron: Economiz
Slide 4: Network role in AI clusters
Network role in AI clusters Frontend fabric Backend fabric Scaling up systems, scaling out clusters • Accelerate model training with Data Parallelism • Split large models across GPUs with Tensor and Pipeline Parallelism • Subdivide complex problems among several models with Mixture of Experts Collec
Slide 5: Collective Operations Terminology
Collective Operations Terminology Broadcast as an Example S Data Data Process Process id = 0 id = 1 • World = group of processes • Rank = id of the process • Collective size (n) = number of ranks Process Process • Data Size (S) = size of memory buffer id = 2 id = 3 • Per rank Data Data • Commonly, t
Slide 6: Types of Collective Operations
Types of Collective Operations • Common types for AI workloads: 0 1 2 3 0 1 2 3 • Broadcast • Gather Gather • AllReduce f • AlltoAll 0 1 2 3 0 1 2 3 • ReduceScatter AllReduce • AllGather • Reduce implies math with data (f) 0 1 2 3 0 1 2 3 • All or Scatter – symmetry AlltoAll 6
Slide 7: Collective Algorithms
Collective Algorithms Implement Collective Operations Example: AllReduce • Ring • Halving-Doubling 3 3 • Double Binary Tree 2 2 • … 1 1 • Software libraries 0 0 • NCCL • RCCL • MSCCL • … 7
Slide 8: Practical challenges
Practical challenges • Synchronized start – from 0 to line rate on all ports • Flow dependencies – latencies accumulate • Low entropy – hard to load balance Unequal Load Balancing • RDMA message bursts – incast Latencies 8 Incast
Slide 9: Key Metrics in Benchmarking
Key Metrics in Benchmarking • Collective Completion Time (CCT) • From first send to last receive across all ranks • Algorithm Bandwidth (algbw) • algbw = S / CCT • Compares algorithm performance • Bus Bandwidth (busbw) • Performance of a bottleneck • Collective-specific formula 9
Slide 10: Extended Collective Metrics
Extended Collective Metrics • Ideal Percentage • Comparison of measured CCT to theoretical minimum • Data Chunk Completion Time (DCT) • Assessing bandwidth fairness • Reporting min, max, P50, and P95 values 10
Slide 11: Visualizing Algorithm Performance
Visualizing Algorithm Performance • AllReduce Halving-Doubling over Clos fabric • Chunks between ranks [0-3] and [4-7] are impacted 11
Slide 12: Open-Source Benchmarking Tools
Open-Source Benchmarking Tools NVIDIA nccl-tests OSU Micro-Benchmarks https://github.com/NVIDIA/nccl/issues/495 https://ulhpc-tutorials.readthedocs.io/en/latest/parallel/mpi/OSU_MicroBenchmarks/ 12
Slide 13: Over 45 Diversified Solutions to Enable 100+ AI Infrastructure Components
Over 45 Diversified Solutions to Enable 100+ AI Infrastructure Components Keysight solutions cover entire workflow Simulation Emulation Validation Testing Manufacturing Network Applications Fabric, AI Data Center Solutions Complementary adjacencies aligned to Systems and Workload Emulation for Netwo
Slide 14: Keysight Solution: KAI Data Center Builder
Keysight Solution Network System Infrastructure Workload Design Design Benchmarks Optimization KAI Data Center Builder Data Center Emulation Hardware Emulation Software Emulation Pre-Silicon Chip Design AI System Design Network Benchmarking & Design 14
Slide 15: Keysight AI Data Center Builder
Keysight AI Data Center Builder Emulate AI Model Training & Inferencing Operations • Emulate AI Workloads • Benchmark AI Network Infrastructure • Co-tune AI Cluster Performance • Reproduce issues seen in production • Validate new designs at system scale High fidelity emulation of GPU servers 15
Slide 16: Keysight AI Applications
Keysight AI Applications Improving Training Performance & Reliability of AI Infrastructure Collective Workload Machine Learning Benchmarks Emulation Stress Test Start Mem Comp Comp Comm Mem Comm Network Utilization Training Time System Reliability 100% 80% 60% 40% 20% Measured Idle 0% 16
Slide 17: Learn More About the Keysight AI Data Center Builder
Learn More About the Keysight AI Data Center Builder Scan QR code to view & download more resources Product Page Solution Brief Whitepaper 17
Slide 18: Keysight
Av KEYSIGHT