All Katas - ML Katas

by: newest upvotes saves

KL Divergence Calculation and Interpretation

this year medium (<30 mins) | probability KL-divergence info-theory statistics VI

The Kullback-Leibler (KL) Divergence (also known as relative entropy) is a non-symmetric measure of how one probability distribution $P$ is different from a second, reference probability...
Numerical Stability: Log-Sum-Exp

this year easy (<10 mins) | optimization deep-learning numerical-stability logarithms probability

When dealing with probabilities, especially in log-space, sums of exponentials can lead to numerical underflow or overflow. For example, computing $\log \left( \sum_i \exp(x_i) \right)$ can be...