KV memory and quantization · Medium

Per-Channel INT8 Quantization

Problem 36 of 50 · Python and NumPy

Problem

Quantize each output column of an R × C matrix symmetrically to integers in [-127,127]. For column c, scale[c] = max(abs(weights[:,c]))/127. If an entire column is zero, use scale 1. Round weights/scale to nearest integer with ties to even, then clip.

Return {"values": R × C integer matrix, "scales": C floating scales}. Zero points are implicitly zero. This symmetric convention deliberately leaves -128 unused.

def quantize_per_channel(weights):

Constraints

  • 1 ≤ R,C ≤ 16; weights contains finite values with magnitude ≤ 10000.
  • Use ties-to-even rounding (Python round or numpy.rint), including negative halfway values.
  • Inputs satisfy these constraints; invalid-input handling is not graded. Use only Python standard library and NumPy.

Examples

Example 1

Arguments: [
  [
    [
      -127,
      0
    ],
    [
      127,
      0
    ]
  ]
]
Expected result: {
  "values": [
    [
      -127,
      0
    ],
    [
      127,
      0
    ]
  ],
  "scales": [
    1,
    1
  ]
}

Column 0 uses scale 1 and represents ±127 exactly; the zero column also uses scale 1.

Example 2

Arguments: [
  [
    [
      0.5
    ],
    [
      -0.5
    ],
    [
      1.5
    ],
    [
      2.5
    ],
    [
      127
    ]
  ]
]
Expected result: {
  "values": [
    [
      0
    ],
    [
      0
    ],
    [
      2
    ],
    [
      2
    ],
    [
      127
    ]
  ],
  "scales": [
    1
  ]
}

Scale is 1, so ties-to-even maps 0.5,-0.5,1.5,2.5 to 0,0,2,2.

Sign in to solve this problem

Browse all coding problems · Learn the foundations