Rotary Position Embeddings

Problem

Rotate adjacent feature pairs (0,1), (2,3), … in each token row. For pair index j starting at zero, angle = positions[t] / base^(2j/D). Transform (a,b) to (a*cos(angle)−b*sin(angle), a*sin(angle)+b*cos(angle)).

Use adjacent-pair ordering, not split-half ordering. Positions are real-valued for this numerical exercise; output keeps shape T × D.

def rotary_embeddings(x, positions, base):

Constraints

• 1 ≤ T ≤ 16; D is even, 2 ≤ D ≤ 16; positions has T nonnegative finite values.
• base > 1; all inputs are finite.
• Inputs satisfy these constraints; invalid-input handling is not graded. Use only Python standard library and NumPy.

Examples

Arguments: [
  [
    [
      1,
      2
    ]
  ],
  [
    0
  ],
  10000
]
Expected result: [
  [
    1,
    2
  ]
]

Arguments: [
  [
    [
      1,
      0
    ]
  ],
  [
    1.5707963267948966
  ],
  10000
]
Expected result: [
  [
    6.123233995736766e-17,
    1
  ]
]