TTFT and Decode Latency

Problem

Compute latency metrics for one token stream using timestamps in seconds. Return {"ttft": first token time minus arrival_time, "itl": consecutive token-time differences, "tpot": mean inter-token latency}.

With zero tokens, ttft and tpot are null (Python None) and itl is []. With one token, ttft is defined but tpot remains null. Coincident token timestamps produce zero intervals. TTFT is excluded from TPOT.

def stream_latencies(arrival_time, token_times):

Constraints

• 0..64 finite token times in nondecreasing order, each ≥ arrival_time.
• arrival_time is finite and nonnegative; all timestamps are at most 100000.
• Inputs satisfy these constraints; invalid-input handling is not graded. Use only Python standard library and NumPy.

Examples

Arguments: [
  10,
  [
    11,
    11.5,
    12.5
  ]
]
Expected result: {
  "ttft": 1,
  "itl": [
    0.5,
    1
  ],
  "tpot": 0.75
}

Arguments: [
  0,
  [
    0.25
  ]
]
Expected result: {
  "ttft": 0.25,
  "itl": [],
  "tpot": null
}