KV memory and quantization · Medium

Affine-Quantized Matrix Multiplication

Problem 37 of 50 · Python and NumPy

Problem

Multiply two affine-quantized integer matrices and return dequantized floating output. Left uses one scalar scale and zero point; right uses a separate scale and zero point for each output column.

Compute sum_k ((left[m,k]−left_zero_point)*left_scale)*((right[k,n]−right_zero_points[n])*right_scales[n]). Do not requantize the output and do not perform narrow INT8 accumulation.

def affine_quantized_matmul(left, right, left_scale, left_zero_point, right_scales, right_zero_points):

Constraints

  • left is M × K, right K × N; dimensions 1..16; integers and zero points lie in [-128,127].
  • Scales are positive finite values; right parameter lists have N entries.
  • Inputs satisfy these constraints; invalid-input handling is not graded. Use only Python standard library and NumPy.

Examples

Example 1

Arguments: [
  [
    [
      1,
      2
    ]
  ],
  [
    [
      3
    ],
    [
      4
    ]
  ],
  1,
  0,
  [
    1
  ],
  [
    0
  ]
]
Expected result: [
  [
    11
  ]
]

With unit scales and zero zero-points, [1,2]·[3,4] = 11.

Example 2

Arguments: [
  [
    [
      3,
      4
    ]
  ],
  [
    [
      5
    ],
    [
      6
    ]
  ],
  0.5,
  2,
  [
    2
  ],
  [
    4
  ]
]
Expected result: [
  [
    5
  ]
]

Left dequantizes to [0.5,1]; right to [2,4], giving 5.

Sign in to solve this problem

Browse all coding problems · Learn the foundations