Tokenization and attention · Medium

Incremental UTF-8 Decoding

Problem 15 of 50 · Python and NumPy

Problem

Decode a stream of UTF-8 byte chunks without corrupting characters split across chunks. Return one string per input chunk, containing only characters that become complete at that chunk. Incomplete bytes must be buffered for the next chunk.

The concatenation of all chunks is valid complete UTF-8. Empty chunks produce empty strings; no chunks returns []. Do not emit replacement characters.

def decode_utf8_chunks(chunks):

Constraints

  • chunks contains 0..32 lists of integers in [0,255]; at most 256 bytes total.
  • The complete byte stream is valid UTF-8, even if individual chunks are not.
  • Inputs satisfy these constraints; invalid-input handling is not graded. Use only Python standard library and NumPy.

Examples

Example 1

Arguments: [
  [
    [
      195
    ],
    [
      169
    ]
  ]
]
Expected result: [
  "",
  "é"
]

Byte 195 starts é but cannot be emitted until byte 169 arrives.

Example 2

Arguments: [
  [
    [
      72,
      105
    ],
    [],
    [
      33
    ]
  ]
]
Expected result: [
  "Hi",
  "",
  "!"
]

ASCII is immediately complete; the empty middle chunk emits nothing.

Sign in to solve this problem

Browse all coding problems · Learn the foundations