KV memory and quantization · Medium

Attention Sinks and Sliding Windows

Problem 35 of 50 · Python and NumPy

Problem

Return the sorted token positions retained by a cache that keeps the first sink_tokens positions and the last window_tokens positions of a length-token sequence. Clip both regions to the existing sequence and remove overlap. The window includes the newest token. A zero-length region contributes no positions.

def retained_cache_positions(length, sink_tokens, window_tokens):

Constraints

  • 0 ≤ length ≤ 128; 0 ≤ sink_tokens,window_tokens ≤ 128.
  • Return positions in increasing order; this is a retention policy, not attention computation.
  • Inputs satisfy these constraints; invalid-input handling is not graded. Use only Python standard library and NumPy.

Examples

Example 1

Arguments: [
  10,
  2,
  3
]
Expected result: [
  0,
  1,
  7,
  8,
  9
]

Keep the two sink positions 0,1 and the latest three positions 7,8,9.

Example 2

Arguments: [
  4,
  3,
  3
]
Expected result: [
  0,
  1,
  2,
  3
]

The prefix and tail overlap and together cover all four positions without duplicates.

Sign in to solve this problem

Browse all coding problems · Learn the foundations