Example 1
Arguments: [
[
[
1,
2
]
],
[
[
3
],
[
4
]
],
1,
0,
[
1
],
[
0
]
]
Expected result: [
[
11
]
]With unit scales and zero zero-points, [1,2]·[3,4] = 11.
KV memory and quantization · Medium
Problem 37 of 50 · Python and NumPy
Multiply two affine-quantized integer matrices and return dequantized floating output. Left uses one scalar scale and zero point; right uses a separate scale and zero point for each output column.
Compute sum_k ((left[m,k]−left_zero_point)*left_scale)*((right[k,n]−right_zero_points[n])*right_scales[n]). Do not requantize the output and do not perform narrow INT8 accumulation.
def affine_quantized_matmul(left, right, left_scale, left_zero_point, right_scales, right_zero_points):Arguments: [
[
[
1,
2
]
],
[
[
3
],
[
4
]
],
1,
0,
[
1
],
[
0
]
]
Expected result: [
[
11
]
]With unit scales and zero zero-points, [1,2]·[3,4] = 11.
Arguments: [
[
[
3,
4
]
],
[
[
5
],
[
6
]
],
0.5,
2,
[
2
],
[
4
]
]
Expected result: [
[
5
]
]Left dequantizes to [0.5,1]; right to [2,4], giving 5.