All Katas - ML Katas

Numerical Gradient Verification

this year medium (<1 hr) | debugging gradient calculus numerical-stability backpropagation

Understanding and correctly implementing backpropagation is crucial in deep learning. A common way to debug backpropagation is using numerical gradient checking. This involves approximating the...

Softmax and its Jacobian

this year hard (>1 hr) | classification gradient calculus numerical-stability softmax

The softmax function is a critical component in multi-class classification, converting a vector of arbitrary real values into a probability distribution. Given an input vector $\mathbf{z} = [z_1,...

Numerical Stability: Log-Sum-Exp

this year easy (<10 mins) | optimization deep-learning numerical-stability logarithms probability

When dealing with probabilities, especially in log-space, sums of exponentials can lead to numerical underflow or overflow. For example, computing $\log \left( \sum_i \exp(x_i) \right)$ can be...