All Katas - ML Katas

Implementing Gradient Clipping

this month easy (<10 mins) | rnn training gradient clipping stability

Implement **gradient clipping** in your training loop. This technique is used to prevent exploding gradients, which can be a problem in RNNs and other deep networks. After the backward pass...

Implementing the Adam Optimizer from Scratch

this month hard (>1 hr) | optimizer adam from scratch gradient

Implement the **Adam optimizer from scratch** as a subclass of `torch.optim.Optimizer`. You'll need to manage the first-moment vector (moving average of gradients) and the second-moment vector...

Differentiating Through a Non-differentiable Function with `torch.autograd.Function`

this month hard (<1 hr) | autograd custom gradient function backprop

Implement a **custom `torch.autograd.Function`** for a non-differentiable operation, such as a custom quantization function. The `forward` method will perform the non-differentiable operation, and...