Slide 1: RISC-V Vector Extension Webinar IV
RISC-V Vector Extension Webinar IV September 29th, 2021 Thang Tran, Ph.D. Principal Engineer
Slide 2: Webinar IV - Agenda
Webinar IV - Agenda • Andes overview • Vector technology background – SIMD/vector concept – Vector processor basic • RISC-V V extension ISA – Basic – CSR • RISC-V V extension ISA – Memory operations – Compute instructions • Sample codes – Matrix multiplication – Loads with RVV versions 0.8 and 1.0 •
Slide 3: Terminology
Terminology • ACE: Andes Custom Extension • ISA: Instruction Set Architecture • CSR: Control and Status Register • GOPS: Giga Operations Per Second • SEW: Element Width (8-64) • GFLOPS: Giga Floating-Point OPS • ELEN: Largest Element Width (32 or 64) • XRF: Integer register file • XLEN: Scalar regis
Slide 4: Sample Codes
Sample Codes Copyright© 2020 Andes Technology Corp. 4
Slide 5: Matrix Multiplication – Issue #495 in RVV Extension Work Group
Matrix Multiplication – Issue #495 in RVV Extension Work Group # void matmulFloat_v( # void matmulFloat_v( # size_t n, // a0: A columns, B rows # size_t 3, // a0: A columns, B rows # size_t m, // a1: A rows # size_t 2, // a1: A rows # size_t p, // a2: B columns # size_t 3, // a2: B columns # const f
Slide 6: Matrix Multiplication – Scalar Coding Style
Matrix Multiplication – Scalar Coding Style 22 vector registers are used for 22 elements of the matrices C 0 A 0 B 0 c0=a0*b0 vfmul.vv v16, v10, v0 0 1 3 c1=a0*b1 vfmul.vv v17, v10, v1 Matrix b0 b1 b2 0 2 6 c2=a0*b2 vfmul.vv v18, v10, v2 Multiplication b3 b4 b5 1 0 1 c3=a3*b0 vfmul.vv v19, v13, v0 b
Slide 7: Vector Scalar-Style Coding
Vector Scalar-Style Coding VLEN=512b, SEW=32b, VLEN/SEW = 512b/32b = 16 elements • Single precision floating point data • 16 sets of matrices can be processed in parallel • Vector multiplication and MACC (6 vfmul and 12 vfmac) Assuming 16 matrices are contiguous in memory, prefetched in L1 data cach
Slide 8: Stride Loads & Stores for Scalar-Style Code
Stride Loads & Stores for Scalar-Style Code vlse32.v v10, x2, x5 Aref 0 vlse32.v v0, x3, x6 Bref 0 vsse32.v v16, x4, x5 Cref 0 addi x2, x2, 4 addi x3, x3, 4 addi x4, x4, 4 vlse32.v v11, x2, x5 Aref 1 vlse32.v v1, x3, x6 Bref 1 vsse32.v v17, x4, x5 Cref 1 addi x2, x2, 4 addi x3, x3, 4 addi x4, x4, 4
Slide 9: Performance of Scalar-Style Codes
Performance of Scalar-Style Codes • Load 6 Arefs = 12*6 = 72 cycles // access 12 32B cache lines • Load 9 Brefs = 17*9 = 153 cycles // access 17 32B cache lines • Store 6 Crefs = 12*6= 54 cycles // access 12 32B cache lines • Vector FP multiplications = 18 cycles // hidden by the store • Total cycle
Slide 10: Matrix Multiplication – Unit Load Coding Style
Matrix Multiplication – Unit Load Coding Style • VLEN=512b, SEW=32b, VLEN/SEW = 512b/32b = 16 elements – LMUL=4, 16x4 = 64 elements – LMUL=8, 16x8 = 128 elements • Number of Cref or Aref per matrix = 6 elements − 10 sets of matrices are 60 elements which fit into VRF with LMUL=4 • Number of Bref per
Slide 11: Unit Load of B elements
Unit Load of B elements • Unit load of 10 consecutive B matrices (10x9=90 elements) C A B 0 0 0 − vrgather to set up 10 sets of 6 B elements = 2, 1, 0, 2, 1, 0 (yellow) in v16 0 1 3 − vslideup v16 to v20 0 2 6 − vrgather to set up 10 sets of 6 B elements = 5, 4, 3, 5, 4, 3 (blue) in v16 1 0 1 1 1 4
Slide 12: Unit Load of A elements
Unit Load of A elements • Unit load of 10 consecutive A matrices (10x6=60 elements) C A B − vrgather to set up 10 sets of 6 A elements = 3, 3, 3, 0, 0, 0 (yellow) in v16 0 0 0 − vrgather to set up 10 sets of 6 A elements = 4, 4, 4, 1, 1, 1 (blue) 0 1 3 0 2 6 − vrgather to set up 10 sets of 6 A eleme
Slide 13: Graphical View of Unit Load Codes
Graphical View of Unit Load Codes VRF b8 31 a5 C A B b7 30 a4 0 0 0 … 29 … 0 1 3 vrgather … 28 … b3 27 a2 0 2 6 b2 26 a1 b1 25 a0 1 0 1 vrgather b0 24 a5 1 1 4 b8 23 a4 b7 22 a3 1 2 7 b6 vslideup 21 a2 2 0 2 b5 20 a1 2 1 5 b4 19 a0 b3 18 2 2 8 b2 17 3 3 0 vrgather b1 16 b0 15 c5 3 4 3 b8 14 c4 3 5 6
Slide 14: Performance of Unit Load Codes
Performance of Unit Load Codes • Load 60 Aref and 90 Bref elements, store 60 Cref elements = 28 cycles of 32B cache lines o Vrgather for Aref = 8 cycles + 4 + 4 = 16 cycles o Vrgather/vslideup for Bref = 13 cycles + 8 + 13 + 13 = 47 cycles • Vector FP multiplications = 12 cycles • Total cycles per l
Slide 15: Larger Matrices – Scalar Style Coding
Larger Matrices – Scalar Style Coding • Same VLEN, SEW=32b, VLMAX=16 – A matrix 6x3, 18 elements – requires 18 vector registers – B matrix 3x6, 18 elements – requires 18 vector registers – C matrix 6x6, 36 elements – requires 36 vector registers – Compute 1 row of A and C elements at a time, 6 itera
Slide 16: Larger Matrices – Unit Load Codes
Larger Matrices – Unit Load Codes • VLEN=512b, SEW=32b, VRF a17 31 VLMAX=16 elements a16 30 … 29 vrgather … 28 a8 a7 27 26 − LMUL=2, 32 elements, 2 a6 a5 25 24 vector registers are used for a4 a3 23 22 each 18 elements of Aref and Bref vmul and vfmac a2 21 vrgather a1 20 a0 19 18 − LMUL=4, 64 elemen
Slide 17: Performances of Larger Matrices
Performances of Larger Matrices Scalar style Unit load Number of static instructions per loop 252 30 Total number of cycles 1259 22* Number of cycles per set of matrices 79 22 Efficiency of register usages 81% 56%** *Vector loads/stores are pipelined between iterations **6x4 or 6x5 matrices are more
Slide 18: RVV Work Group, Issue #362, Codes Based on RVV 0.8
RVV Work Group, Issue #362, Codes Based on RVV 0.8 vsetvli t0, zero, e32, m8 // VLEN=512b, LMUL=8, SEW=32b, ELEN=4096b vlb.v v0, t0; (RVV 0.8) add t0, t0, imm; vadd.vx v0, v0, t1; lfw f0, t2; Loop of 8 instructions, add t2, t2, imm; Load element = 8b INT, extension to 32b vfcvt.f.x.v v0, v0; Operati
Slide 19: Vector Processor Pipeline Example
Vector Processor Pipeline Example VLEN=64B 8 cycles 8 cycles 8 cycles Time 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 vadd v0, v0, t1 va va va vadd v1, v1, t1 va va va vadd v2, v2, t1 va va va vadd v3, v3, t1 va va va vadd v4, v4, t1 va va va vadd v5, v5, t1 va va va vadd v6, v6,
Slide 20: Vector Processor Performance
Vector Processor Performance • Number of instructions and micro-ops in 8 cycles – 3 vector arithmetic instructions = 24 micro-ops (vadd, vfcvt, vfmac) – 1 FP load – 3 scalar instructions + 1 vector load to data cache – Performance at 1 GHz = 64 GOPS or 48 GFLOPS for SEW=32b – Performance at 1 GHz =
Slide 21: RVV Work Group, Issue #362, Codes Based on RVV 0.9
RVV Work Group, Issue #362, Codes Based on RVV 0.9 vle8.v v6, t0; add t0, t0, imm; vsext.v4 v0, v6 vadd.vx v0, v0, t1; Loop of 9 instructions, lfw f0, t2; Load element = 8b INT, add t2, t2, imm; Vector extension to 32b, vfcvt.f.x.v v0, v0; Operation on FP 32b vfmacc.vf v8, v0, f0; bne t0, t3, imm •
Slide 22: AndesCore™ NX27V introduction
AndesCore™ NX27V introduction Copyright© 2020 Andes Technology Corp. 22
Slide 23: Andes NX27V Design Objectives
Andes NX27V Design Objectives • Vector Processor is implemented as part of the RISC-V CPU • Configurability: necessary for IP to attract wide range of customers, 128b to 512b • Scalability: necessary for next generation • Low power and area: necessary for embedded market • Performance: out-of-order
Slide 24: NX27V: Powerful Vector Processor
NX27V: Powerful Vector Processor ◆An efficient Scalar Unit ◆A powerful decoupled OOO VPU – Data formats or SEW • Standard: int8/16/32/64 and fp16/32/64 • Andes extended: bfloat16, and int4 – RVV instructions start execution after retired – Functional units: parallel, most pipelined/chainable – VLEN
Slide 25: NX27V: ACE Streaming Port (ASP)
NX27V: ACE Streaming Port (ASP) ◼ A common scenario in SoC with HW accelerators: ⚫ DMA-equipped HW engine accelerates regular/structured computations (e.g. CNN, ME, FFT) ⚫ Efficient & programmable pre/post-processing is needed → algorithm innovation/evolution Streaming Port HW Engine with Scalar pip
Slide 26: RVV Tools and Performance
RVV Tools and Performance ◼ Standard tools in AndeSight™ ➢ AndeSim™: Cycle simulator ➢ Compiler: intrinsic functions RVV Functions NX27V Speedup1 ➢ Assembler ➢ Debugger and ICE F32 basic function 23x ➢ Computation libraries ◼ Advanced tools: F32 32x32 GEMM 43x ➢ LLVM IR support for AI compilers Q7 C
Slide 27: NX27V Performance Data
NX27V Performance Data Features NX27V VLEN/SIMD 256/256 512/256 512/512 ELEN (bit) 32 (int+fp) Max Frequency (worst case)1 All frequencies >=1.2G Gate Count (gates)1 From 1.6M to 2.6M Dynamic Power (uW/MHz)1 < 17 F32 GEMM 32X32 (cycle) Around 6,200 Around 4,800 Around 3,400 Note 1: 7nm mixed VT 240H
Slide 28: Andes NX27V vs. Competitions
Andes NX27V vs. Competitions Andes NX27V CAxx CMxx Architecture RVV/Andes VPU Popular SIMD Hxxx Vector registers 32 32 8 Vector Length Up to 512b 128b 128b SIMD width Up to 512b/cycle 128b/cycle 64b/cycle LMUL Yes None None Chaining Yes Not applicable Yes Custom extension ACE No No Streaming Ports Y
Slide 29: Core/System Performance Comparison
Core/System Performance Comparison CPU A64FX* NX27V** Vector ISA ARM SVE RISC-V RVV Technology (nm) 7 7 Core sustain Perf 16b (GFLOPS) ~56 96 Core Peak Perf 16b (GOPS) 230 320 # of cores 48 48** System (TFLOPS) ~2.7 ~4.6 Memory BW (GB/s) 1024 3072 *Fujitsu presentation at Hot Chip 2018. **Assume the
Slide 30: Development Tools
Development Tools ❖Standard tools in AndeSight™ IDE: – AndeSim: Near-cycle accurate simulator – Compiler (intrinsic functions and inline assembly) – Assembler – Debugger and ICE – Computation library ❖Advanced tools: – AI compiler support thru LLVM – AndesClarity pipeline visualizer and analyzer ❑ P
Slide 31: Clarity: NX27V Pipeline View
Clarity: NX27V Pipeline View Copyright© 2020 Andes Technology Corp. 31
Slide 32: Summary
Summary • Andes NX27V vector processor – The world first commercial RISC-V vector processor, VLEN = 512b/256b/128b are available (announced in December 2019) – Five customer projects … – High performance in HPC and AI applications – Flexible VPU configurations to enable a wide range of applications
Slide 33: Find Latest Trends
, Find Latest Trends _ Follow Andes
Slide 34: Thank you!
Thank you!