All Katas - ML Katas

Implementing a Custom `nn.Module` for a Gated Recurrent Unit (GRU)

this month medium (<1 hr) | rnn gru custom module recurrent

Implement a **custom GRU cell** as a subclass of `torch.nn.Module`. Your implementation should handle the reset gate, update gate, and the new hidden state computation from scratch, using...

Implementing a Custom Learning Rate Scheduler

this month medium (<1 hr) | training custom scheduler learning rate

Implement a **custom learning rate scheduler** that follows a cosine annealing schedule. The learning rate starts high and decreases smoothly to a minimum value, then resets and repeats. Your...

Transfer Learning with a Pre-trained Model

this month medium (<1 hr) | cnn transfer learning resnet fine-tuning

Fine-tune a **pre-trained model** (e.g., `resnet18` from `torchvision.models`) on a new, small image classification dataset (e.g., `CIFAR-10`). You'll need to freeze the weights of the initial...

Implementing a Simple Attention Mechanism

this month medium (<1 hr) | rnn attention mechanism seq2seq weights

Implement a **simple attention mechanism** for a sequence-to-sequence model. Given a sequence of encoder outputs and a single decoder hidden state, your attention module should compute attention...

Implementing Weight Initialization Schemes

this month medium (<1 hr) | training initialization weight he xavier

Implement **different weight initialization schemes** (e.g., Xavier/Glorot, He) for a simple neural network. Create a function that iterates through a model's parameters and applies a chosen...

Custom Dataset for CSV Data

this month medium (<1 hr) | pytorch dataset dataloader tabular data

Write a PyTorch `Dataset` class that loads data from a CSV file containing tabular data (features + labels). Requirements: - Use `pandas` to read the CSV. - Convert features and labels to tensors....

Implement Dropout Manually

this month medium (<1 hr) | pytorch implementation dropout regularization

Implement dropout as a function `my_dropout(x, p)`: - Zero out elements of `x` with probability `p`. - Scale survivors by $$1/(1-p)$$. - Ensure deterministic behavior when `torch.manual_seed` is...

Custom Activation Function

this month medium (<1 hr) | pytorch implementation activations mlp

Define a custom activation function called `Swish`: $$f(x) = x \cdot \sigma(x)$$. - Implement it as a PyTorch `nn.Module`. - Train a small MLP on random data with it. - Compare with ReLU...

Visualize Training with TensorBoard

this month medium (<1 hr) | pytorch training tensorboard visualization

Integrate TensorBoard into a training loop: - Log training loss and validation accuracy. - Add histograms of weights and gradients. - Write a few sample images. Open TensorBoard and verify logs.

Implement Early Stopping

this month medium (<1 hr) | pytorch training basics early stopping

Add early stopping to a training loop: - Monitor validation loss. - Stop training if no improvement after 5 epochs. - Save best model checkpoint. Demonstrate on MNIST subset.