Slide 1: RISC-V Vector Extension Webinar III
RISC-V Vector Extension Webinar III August 31st, 2021 Thang Tran, Ph.D. Principal Engineer
Slide 2: Webinar III - Agenda
Webinar III - Agenda • Andes overview • Vector technology background – SIMD/vector concept – Vector processor basic • RISC-V V extension ISA – CSR • RISC-V V extension ISA – Memory operations – Compute instructions – Permute instructions • Sample codes – Matrix multiplication – Loads with RVV versio
Slide 3: Terminology
Terminology • ACE: Andes Custom Extension • ISA: Instruction Set Architecture • CSR: Control and Status Register • GOPS: Giga Operations Per Second • SEW: Element Width (8-64) • GFLOPS: Giga Floating-Point OPS • ELEN: Largest Element Width (32 or 64) • XRF: Integer register file • XLEN: Scalar regis
Slide 4: LMUL – Registers Grouping
LMUL – Registers Grouping LMUL=1 LMUL=4 v31 v28 v30 v24 v29 … v28 v4 … LMUL=2 v0 … v30 … v28 … … LMUL=8 v3 … v24 v2 … v16 v1 v2 v8 v0 v0 v0 • LMUL=2, instructions with odd register are illegal instructions • LMUL=4, vector registers are incremented by 4, else illegal instructions • LMUL=8, only v0,
Slide 5: RISC-V V Extension ISA Memory Operations
RISC-V V Extension ISA Memory Operations Copyright© 2020 Andes Technology Corp. 5
Slide 6: Memory Operations
Memory Operations • Load/store operations move groups of data between the VRF and memory – Andes implements unaligned addressing, no restriction • Four basic types of addressing – Unit stride: fetch consecutive elements in contiguous memory – Non-unit or constant stride: elements are in stride memor
Slide 7: Load Data Alignment
Vector Load Data Alignment Andes processor handles all unaligned addresses Byte 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 First package 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Second package 31 30 29 28 27 26 2
Slide 8: Instruction Format – Load/Store
Instruction Format – Load/Store 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 VL* unit-stride nf mw mop vm lumop rs1 width vd 0000111 VLS* strided nf mw mop vm rs2 rs1 width vd 0000111 VLX* indexed nf mw mop vm vs2 rs1 width vd 0000111 VS* unit-stride nf mw mo
Slide 9: Vector load/store, different EMUL
Vector load/store, different EMUL • EMUL is the effective LMUL • SEW=16, LMUL=1, VLEN=512b – elements=512/16=32, all vector instructions operate on 32 elements vle16.v v1, (t0) # Load a vector of sew=16, lmul=1, load to v1 vle32.v v2, (t1) # Load a vector of sew=32, emul=2, load to v2 and v3 vwadd.w
Slide 10: Unit-Stride Load/Store Variances
Unit-Stride Load/Store Variances • Load Fault-only-First: exception only for first element – Set vl=faulted element, no exception – Use for exit condition of loop – Example: the unit load fetches to the end of the TLB page, all subsequent vector operations will be for the valid elements that were fe
Slide 11: Example: strncpy, Fault-only-First Load
Example: strncpy, Fault-only-First Load strncpy: mv a3, a0 // Copy dst loop: vsetvli x0, a2, e8, m8, ta, ma // Vectors of bytes, VLEN=512b vle8ff.v v8, (a1) // Fault-only-First load to 8 vector registers vmseq.vi v1, v8, 0 // Flag zero bytes (mask) csrr t1, vl // Get number of bytes fetched Note: ta
Slide 12: Stride Gather/Scatter Memory Access
Stride Gather/Scatter Memory Access • Unit stride continuous memory access • Constant stride: gather scatter (stride can be negative or zero) – Support store memory overlapped due to stride=0 or stride<SEW 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Unit 0 1 2 3 unit stride 0 1 2 3 4 5 6 7 8 9 10 11
Slide 13: Index Gather/Scatter Memory Access
Index Gather/Scatter Memory Access Base Address &data[0] Load Index vector 7 12 8 2 vs2 Gather 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Base Address &data[0] Store Index vector 5 8 0 15 vs2 3 2 1 0 vd/vs3 Scatter 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Note: - The indices are absolute and not accum
Slide 14: Segment Unit Stride Load/Store
Segment Unit Stride Load/Store • Multiple vector registers load/store, NF*EMUL <= 8 • Equivalent to 4 stride load/store micro-ops in below example • Fault-only-first is allowed Unit load/store with NF=4, LMUL=1 Load 0 Load 1 Load 2 0 1 2 3 4 5 6 7 8 9 10 11 Memory Vector Registers V31 . . . . V3 V2
Slide 15: Segment Stride Load/Store, LMUL=2
Segment Stride Load/Store, LMUL=2 • Multiple vector registers load/store, NF*EMUL <= 8 • Zero and negative strides are allowed Unit load/store with NF=2, LMUL=2 Load 0 Load 1 Load 2 0 1 2 3 4 5 6 7 8 9 10 11 Memory Vector Registers V31 . . . . Second stride load/store, base address=1 V3 V2 V1 V0 Fir
Slide 16: Segment Index Load/Store
Segment Index Load/Store • Multiple vector registers load/store, NF*EMUL <= 8 – Overlapping of store data to memory – NF number of index load/store micro-ops, the index is incremented by SEW Base Address &data[0] Index vector 2 8 12 7 Memory 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Vector Registers
Slide 17: Matrix Multiplication Loads Example
Matrix Multiplication Loads Example a0 a1 a2 a3 b0 b1 b2 b3 a4 a5 a6 a7 b4 b5 b6 b7 b8 b9 ba bb matrix multiplication bc bd be bf Memory bf be bd bc bb ba b9 b8 b7 b6 b5 b4 b3 b2 b1 b0 bf be bd bc bb ba b9 b8 b7 b6 b5 b4 b3 b2 b1 b0 second 4x4 matrix first 4x4 matrix NF=4 segment loads VRF 31 30 29
Slide 18: RISC-V V Extension ISA Compute Instruction Format
RISC-V V Extension ISA Compute Instruction Format Copyright© 2020 Andes Technology Corp. 18
Slide 19: Arithmetic Instruction Overview
Vector Arithmetic Instruction Overview • vop.vv vd, vs2, vs1, vm // vector-vector, vd[i]=vs2[i] op vs1[i] • vop.vx vd, vs2, rs1, vm // vector-scalar, vd[i]=vs2[i] op x[rs1] • vop.vi vd, vs2, imm, vm // vector-imm, vd[i]=vs2[i] op imm • vfop.vv vd, vs2, vs1, vm // vector-vector, vd[i]=vs2[i] fop vs1[
Slide 20: Compute Instruction Format
Compute Instruction Format 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 OPIVV vector-vector funct6 m vs2 vs1 000 vd 1010111 OPFVV vector-vector funct6 m vs2 vs1 001 vd/rd 1010111 OPMVV vector-vector funct6 m vs2 vs1 010 vd/rd 1010111 OPIVI vector-immed funct6
Slide 21: Widening Instructions
Vector Widening Instructions • Widening: double width − vwop.v{v,x} vd, vs2, vs1/rs1 // 2*SEW = sign/zero(SEW) op sign/zero(SEW) − vwop.w{v,x} vd, vs2, vs1/rs1 // 2*SEW = 2*SEW op sign/zero(SEW) − vfwop.v{v,f} vd, vs2, vs1/rs1 // 2*SEW = SEW op SEW − vfwop.w{v,f} vd, vs2, vs1/rs1 // 2*SEW = 2*SEW op
Slide 22: Widening Instructions – Extension
Vector Widening Instructions – Extension • Vector integer extension − V{z,s}ext.vf{2,4,8} vd, vs2 // Zero/sign extend to SEW − The result is the same number of elements set by vtype − Example: SEW=32b, then only V{z,s}ext.vf{2,4} are legal instructions Copyright© 2020 Andes Technology Corp. 22
Slide 23: Shifting of Source Operands for Double-Width
Shifting of Source Operands for Double-Width • The data are extended and folded into the source data from VRF before sending to execution – Cycle 1: operate on full-width data and write back to v2 shift data from left-half and extension to full width – Cycle 2: operate on full-width data and write b
Slide 24: Narrow Instructions
Vector Narrow Instructions • Narrowing: from double-width – vnop.w{v,x,i} vd, vs2, vs1/rs1 // SEW = 2*SEW op SEW – Logical and arithmetic right shift operations Example of operations without changing vtype, LMUL=1, SEW=16b, 32 elements in all vector instruction operations: vle16.v v1, (t0) // Load t
Slide 25: Shifting of Source Operands for Narrow-Width
Shifting of Source Operands for Narrow-Width • The data are extended and folded into the source data from VRF before sending to execution – Cycle 1: operate on full-width data – Cycle 2: Shift result data to write back to right-half of v0 Pipeline in after the right half to write to v3 16b 16b 16b 1
Slide 26: RISC-V V Extension ISA Compute Instructions
RISC-V V Extension ISA Compute Instructions Copyright© 2020 Andes Technology Corp. 26
Slide 27: Integer Arithmetic Instructions
Vector Integer Arithmetic Instructions Vector Integer Add, Subtract, Reverse Subtract (including Double Width) Vector Integer Extension Vector Integer Add-with-Carry / Subtract-with-Borrow Vector Bitwise Logical Instructions Vector Bit Shift Instructions (including Narrow Width) Vector Integer Compa
Slide 28: Arithmetic with Carry Instructions
Vector Arithmetic with Carry Instructions vadc.vv vd, vs2, vs1, vm=0 // vd[i] = vs2[i] + vs1[i] + v0[i].mask vmadc.vvm vd, vs2, vs1, vm=0 // vd[i].mask = carry_out(vs2[i] + vs1[i] + v0[i].mask) • The carry-in is in v0 same as mask register • Vector instructions oriented to implementation : − Single
Slide 29: MAC Instructions
Vector MAC Instructions v(w)madd.vv vd, vs2, vs1, vm // vd[i] = vs2[i] + (vs1[i] * vd[i]) v(w)madd.vx vd, vs2, rs1, vm // vd[i] = vs2[i] + (rs1 * vd[i]) v(w)macc.vv vd, vs2, vs1, vm // vd[i] = vd[i] + (vs1[i] * vs2[i]) v(w)macc.vx vd, vs2, rs1, vm // vd[i] = vd[i] + (rs1 * vs2[i]) v(w)msub and v(w)m
Slide 30: Merge Instructions
Vector Merge Instructions vmerge.vx vd, vs2, rs1, vm=0 // vd[i] = vm[i] ? rs1 : vs2[i] vmerge.vv vd, vs2, vs1, vm=0 // vd[i] = vm[i] ? vs1[i] : vs2[i] vmerge.vi vd, vs2, imm, vm=0 // vd[i] = vm[i] ? imm : vs2[i] • Each element uses vm[i] to select the input element Copyright© 2020 Andes Technology C
Slide 31: Integer Move Instructions
Vector Integer Move Instructions vmv.v.{v,x,i} vd, {vs1, rs1, imm}, vm=1 • Same encoding as with Vector Merge Instructions with vm=1 and vs2=v0 • {x,i} splat a scalar register or immediate to all elements • vs1=vd is NOP • Other vs2 values are reserved Copyright© 2020 Andes Technology Corp. 31
Slide 32: Fixed-Point Arithmetic Instructions
Vector Fixed-Point Arithmetic Instructions Vector Saturating Add and Subtract (set CSR vxsat bit) Vector Averaging Add and Subtract (overflow is ignored) – Add/subtract, then shift right by 1, round off according to CSR vxrm bit Vector Fractional Multiply with Rounding and Saturation Vector Scaling
Slide 33: Fractional Multiply with Rounding and Saturation
Vector Fractional Multiply with Rounding and Saturation vsmul.v{v,x} vd, vs2, {vs1,rs1}, vm – Double-width multiplication vs2[i]*{vs1[i], rs1} – Shift right by SEW-1 – Roundoff_signed according to CSR vxrm bit – Saturate the result to SEW bits, set CSR vxsat bit if saturated – vd[i] = clip(roundoff_
Slide 34: Scale-Shift and Clip Instructions
Vector Scale-Shift and Clip Instructions vssrl.v{v/x/i} vd, vs2, {vs1,rs1,imm}, vm // scale-shift right logical – vd[i] = round_unsigned((vs2[i] + rnd) >> {vs1[i],rs1,imm}) vssra.v{v/x/i} vd, vs2, {vs1,rs1,imm}, vm // scale-shift right arithmetic – vd[i] = round_signed((vs2[i] + rnd) >> {vs1[i],rs1,
Slide 35: Floating-Point Arithmetic Instructions
Vector Floating-Point Arithmetic Instructions Vector FP Add, Subtract, Reverse Subtract (including Double Width) Vector FP Multiply (including Double Width) Vector FP Divide, Reverse Divide Vector FP Multiply-Add/Sub, Multiply-Accumulate (including Double Width) Vector FP Square-Root, Reciprocal Squ
Slide 36: Reduction Instructions
Vector Reduction Instructions Vector Integer Reduction Instructions Reduction sum (including Double Width) Reduction min/max (including signed and unsigned) Reduction bitwise AND, OR, XOR Vector FP Reduction Instructions Reduction ordered sum (including Double Width) ((((vs1[0] + vs2[0]) + vs2[1]) +
Slide 37: Reduction Sum
Vector Reduction Sum • Reduction of VLEN/SEW=256-bit/16-bit=16 elements • LMUL>1 can be pipelined SWPL = 64-bit 64-bit 64-bit 64-bit SEW = 16b 16b 16b 16b 16b 16b 16b 16b 16b 16b 16b 16b 16b 16b 16b 16b V3-256b E15 E14 E13 E12 E11 E10 E9 E8 E7 E6 E5 E4 E3 E2 E1 E0 V1-256b E15 E14 E13 E12 E11 E10 E9
Slide 38: Mask Instructions
Vector Mask Instructions Vector Mask-Register Logical Instructions AND, NAND, AND-NOT, OR, NOR, OR-NOT, XOR, XNOR Vector mask population count vpopc vfirst find-first-set mask bit vmsbf.m set-before-first mask bit vmsif.m set-including-first mask bit vmsof.m set-only-first mask bit Vector Iota Instr
Slide 39: Mask Logical
Vector Mask Logical • Mask logical instructions, vd = vs2 (logical-op) vs1 – vmmv.m = vmand.mm vd, vs, vs // copy mask register – vmclr.m = vmxor.mm vd, vd, vd // clear mask register – vmset.m = vmxnor.mm vd, vd, vd // set mask register – vmnot.m = vmnand.mm vd, vs, vs // invert mask bits • vl and v
Slide 40: Mask Instructions
Vector Mask Instructions [VLMAX-1] … [8] [7] [6] [5] [4] [3] [2] [1] [0] vs2 1 … 0 0 1 1 0 1 0 0 0 Mask layout as with v0 mask register vstart=0, except for vid rd + 14 vpopc - x[rd] = sum_i( vs2[i] && v0[i] ) rd 3 vmfirst - element index of first mask bit vd 0 … 0 0 0 0 0 0 1 1 1 vmsbf - set before
Slide 41: RISC-V V Extension ISA Permute Instructions
RISC-V V Extension ISA Permute Instructions Copyright© 2020 Andes Technology Corp. 41
Slide 42: Permutation Instructions
Vector Permutation Instructions Integer Scalar Move Instructions – between vector element 0 and x[rd/rs1] Floating-Point Scalar Move Instructions – between vector element 0 and f[rd/rs1] Vector Slide Instructions Vector Register Gather Instructions Vector Compress Instruction Whole Vector Register M
Slide 43: Slide Instructions
Vector Slide Instructions • vslideup/down – move elements up and down a vector − vslide1up.vx vd, vs2, rs1, vm // vd[0]=x[rs1], vd[i+1] = vs2[i] − vslideup.vx vd, vs2, rs1, vm // vd[i+rs1] = vs2[i] − vslideup.vi vd, vs2, uimm, vm // vd[i+uimm] = vs2[i] − vslide1down.vx vd, vs2, rs1, vm // vd[i] = vs
Slide 44: Slideup Instruction
Vector Slideup Instruction • No source and destination registers overlapping, destination register can be written before the source register is read (RAW) − vslideup.vx vd, vs2, rs1, vm // vd[i+rs1] = vs2[i] − vslideup.vi vd, vs2, uimm, vm // vd[i+uimm] = vs2[i] − vslide1up.vx vd, vs2, rs1, vm // vd
Slide 45: Slidedown Instruction
Vector Slidedown Instruction • Source and destination registers overlapping is okay, source register is read before the destination register is written − vslidedown.vx vd, vs2, rs1, vm // vd[i] = vs2[i+rs1] − vslidedown.vi vd, vs2, uimm, vm // vd[i] = vs2[i+uimm] − vslide1down.vx vd, vs2, rs1, vm //
Slide 46: Register Gather Instructions
Vector Register Gather Instructions • vrgather – read elements from a source vector with index register to write destination vector – vrgather.vv vd, vs2, vs1, vm // vd[i] = vs2[vs1[i]] – vrgatherei16.vv vd, vs2, vs1, vm // vd[i] = vs2[vs1[i]], vs1 is 16b – vrgather.vx vd, vs2, rs1, vm // vd[i] = vs
Slide 47: Compress Instructions
Vector Compress Instructions • vcompress – shift the unmask elements from source vector to destination register − vcompress.vm vd, vs2, vs1 // vd[i] = Compress into vd elements of vs2 where vs1(LSB) is enabled − vm=1 (inst[25]), else illegal instruction [VLMAX-1] … [8] [7] [6] [5] [4] [3] [2] [1] [0
Slide 48: Find Latest Trends, Follow Andes
, Find Latest Trends _ Follow Andes
Slide 49: Thank you!
Thank you!