ML Katas

Gradient Clipping Example

medium (<30 mins) pytorch rnn training gradients
this year by E

Write code to:

  1. Train a small RNN on dummy data.
  2. Add gradient clipping using torch.nn.utils.clip_grad_norm_.
  3. Print gradient norms before and after clipping.

Show that exploding gradients are controlled.