ML Katas

Implement gradient clipping in your training loop. This technique is used to prevent exploding gradients, which can be a problem in RNNs and other deep networks. After the backward pass (loss.backward()), you will call torch.nn.utils.clip_grad_norm_ on your model's parameters with a maximum norm value. This ensures that the magnitude of the gradients does not exceed a certain threshold.

Verification: Train a deep RNN without gradient clipping on a long sequence. You'll likely observe NaN values in the loss or very large, unstable losses. Then, introduce gradient clipping and rerun the training. The training process should become stable, and the loss should decrease smoothly, without exploding.