Example 1
Arguments: [
[
[
195
],
[
169
]
]
]
Expected result: [
"",
"é"
]Byte 195 starts é but cannot be emitted until byte 169 arrives.
Tokenization and attention · Medium
Problem 15 of 50 · Python and NumPy
Decode a stream of UTF-8 byte chunks without corrupting characters split across chunks. Return one string per input chunk, containing only characters that become complete at that chunk. Incomplete bytes must be buffered for the next chunk.
The concatenation of all chunks is valid complete UTF-8. Empty chunks produce empty strings; no chunks returns []. Do not emit replacement characters.
def decode_utf8_chunks(chunks):Arguments: [
[
[
195
],
[
169
]
]
]
Expected result: [
"",
"é"
]Byte 195 starts é but cannot be emitted until byte 169 arrives.
Arguments: [
[
[
72,
105
],
[],
[
33
]
]
]
Expected result: [
"Hi",
"",
"!"
]ASCII is immediately complete; the empty middle chunk emits nothing.