In production LLMs, logit values can be very large. Naively computing exp(800) causes **float overflow** (inf). The **log-sum-exp trick** fixes this.
`exp(1000)` → overflow (inf)
stable_softmax(x) = softmax(x - max(x))This is mathematically identical (shift invariance) but numerically safe.
stable_softmax([1000, 1001, 1002]) should NOT overflow → [0.09003, 0.24473, 0.66524]
log_softmax(x) = x - max(x) - log(Σ exp(x - max(x)))Round to **5 decimal places**.
Similar Problems
Test Cases (2 visible · 2 hidden)
Case 1: Large values no overflow
Input: stable_softmax([1000,1001,1002])
Expected: [0.09003, 0.24473, 0.66524]
Case 2: Same as regular softmax
Input: stable_softmax([1.0,2.0,3.0])
Expected: [0.09003, 0.24473, 0.66524]
⌘↵ Run · ⌘⇧↵ Submit