Example 1
Arguments: [
[
[
0,
0
]
],
[
1
],
-100
]
Expected result: 0.6931471805599453Two equal logits assign probability 1/2 to label 1, so loss is log(2).
Tensor and numerical foundations · Medium
Token Cross Entropy with Ignored Labels
Problem 6 of 50 · Python and NumPy
Problem
Return the mean negative log probability of the labeled token across non-ignored rows. Labels equal to ignore_index contribute neither loss nor denominator. If every label is ignored, return 0.0.
Compute log-sum-exp stably; very unlikely target probabilities must not underflow before taking a logarithm.
def token_cross_entropy(logits, labels, ignore_index):Constraints
- logits is T × V with 1 ≤ T,V ≤ 32; labels has T integers.
- A label is a valid vocabulary index or ignore_index; logits are finite in [-10000,10000].
- Inputs satisfy these constraints; invalid-input handling is not graded. Use only Python standard library and NumPy.
Examples
Example 2
Arguments: [
[
[
1000,
-1000
],
[
0,
0
]
],
[
-100,
0
],
-100
]
Expected result: 0.6931471805599453The first row is ignored; the remaining row contributes log(2).