All Katas - ML Katas

Softmax and its Jacobian

this year hard (>1 hr) | classification gradient calculus numerical-stability softmax

The softmax function is a critical component in multi-class classification, converting a vector of arbitrary real values into a probability distribution. Given an input vector $\mathbf{z} = [z_1,...

Backpropagation for a Single-Layer Network

this year hard (>1 hr) | gradient deep-learning calculus backpropagation neural-networks

Backpropagation is the cornerstone algorithm for training neural networks. It efficiently calculates the gradients of the loss function with respect to all the weights and biases in the network by...

Implementing the Adam Optimizer from Scratch

this year hard (>1 hr) | optimizer adam from scratch gradient

Implement the **Adam optimizer from scratch** as a subclass of `torch.optim.Optimizer`. You'll need to manage the first-moment vector (moving average of gradients) and the second-moment vector...

Differentiating Through a Non-differentiable Function with `torch.autograd.Function`

this year hard (<1 hr) | autograd custom gradient function backprop

Implement a **custom `torch.autograd.Function`** for a non-differentiable operation, such as a custom quantization function. The `forward` method will perform the non-differentiable operation, and...