All Katas - ML Katas

by: newest upvotes saves

The Elegant Gradient of Softmax-Cross-Entropy

this year hard (>1 hr) | Backpropagation Cross-Entropy Softmax Gradients

One of the most satisfying derivations in deep learning is the gradient of the combined Softmax and Cross-Entropy loss. For a multi-class classification problem with $K$ classes, given true labels...
Cross-Entropy: A Measure of Surprise

this year medium (<30 mins) | Loss Functions Cross-Entropy Information Theory

Cross-entropy loss is fundamental for classification tasks. Let's build some intuition for its formulation. 1. **Definition**: For a binary classification problem, the binary cross-entropy (BCE)...