LayerNorm

Problem

Center and normalize each activation row, then apply a learned affine transform. Use population variance mean((x−mean(x))²), not sample variance. Compute y = (x−mean)/sqrt(variance+eps)*weight+bias.

def layer_norm(x, weight, bias, eps):

Constraints

• x is N × D; weight and bias have D entries; 1 ≤ N,D ≤ 16.
• eps ≥ 0 and variance+eps > 0 for every row.
• Inputs satisfy these constraints; invalid-input handling is not graded. Use only Python standard library and NumPy.

Examples

Arguments: [
  [
    [
      1,
      3
    ]
  ],
  [
    1,
    1
  ],
  [
    0,
    0
  ],
  0
]
Expected result: [
  [
    -1,
    1
  ]
]

Arguments: [
  [
    [
      2,
      2
    ]
  ],
  [
    3,
    4
  ],
  [
    1,
    -1
  ],
  1
]
Expected result: [
  [
    1,
    -1
  ]
]