Gradient Clipping

Prevents exploding gradients by scaling down large gradient vectors. Standard technique in RNN and Transformer training.

Algorithm

1. Compute L2 norm: `||g|| = sqrt(Σg_i²)`

2. If `||g|| > max_norm`: scale gradient by `max_norm / ||g||`

Example

gradient_clip([3.0,4.0], max_norm=2.0)
# ||g|| = 5.0, scale = 2/5 = 0.4
→ [1.2, 1.6]

Round to **5 decimal places**.