All Katas - ML Katas

Custom Dataset Class

this month medium (<30 mins) | dataloader nn datasets

Create a custom PyTorch `Dataset` for pairs of numbers and their sum. ### Problem Implement a dataset where each sample is `(x, y, x+y)`. - **Input:** A list of tuples `(x, y)`. - **Output:** For...

Implement a Simple MLP

this month medium (<30 mins) | mlp nn forward pass

Build and run a minimal Multi-Layer Perceptron (MLP) using `torch.nn`. ### Problem Construct a 2-layer MLP with ReLU activation for input of size 10 and output of size 2. - **Input:** Tensor of...

Implement a Custom Loss Function

this month medium (<30 mins) | pytorch loss functions regression implementation

Create a custom loss function called `MeanAbsolutePercentageError` (MAPE) in PyTorch. It should: 1. Take predictions and targets as input tensors. 2. Compute $$\frac{1}{n} \sum_i \frac{|y_i -...

Custom Dataset for CSV Data

this month medium (<1 hr) | pytorch dataset dataloader tabular data

Write a PyTorch `Dataset` class that loads data from a CSV file containing tabular data (features + labels). Requirements: - Use `pandas` to read the CSV. - Convert features and labels to tensors....

Gradient Clipping Example

this month medium (<30 mins) | pytorch rnn training gradients

Write code to: 1. Train a small RNN on dummy data. 2. Add gradient clipping using `torch.nn.utils.clip_grad_norm_`. 3. Print gradient norms before and after clipping. Show that exploding gradients...

Implement Dropout Manually

this month medium (<1 hr) | pytorch implementation dropout regularization

Implement dropout as a function `my_dropout(x, p)`: - Zero out elements of `x` with probability `p`. - Scale survivors by $$1/(1-p)$$. - Ensure deterministic behavior when `torch.manual_seed` is...

Custom Activation Function

this month medium (<1 hr) | pytorch implementation activations mlp

Define a custom activation function called `Swish`: $$f(x) = x \cdot \sigma(x)$$. - Implement it as a PyTorch `nn.Module`. - Train a small MLP on random data with it. - Compare with ReLU...

Weight Initialization Techniques

this month medium (<30 mins) | pytorch deep learning training initialization

Initialize a neural network's weights using different schemes: - Xavier initialization. - Kaiming initialization. Show histograms of weight distributions before and after initialization.

Custom Collate Function

this month medium (<30 mins) | pytorch dataloader sequences collate

Write a custom `collate_fn` for `DataLoader` that pads variable-length sequences with zeros. Use `torch.nn.utils.rnn.pad_sequence`. Test by batching random-length tensors.

Visualize Training with TensorBoard

this month medium (<1 hr) | pytorch training tensorboard visualization

Integrate TensorBoard into a training loop: - Log training loss and validation accuracy. - Add histograms of weights and gradients. - Write a few sample images. Open TensorBoard and verify logs.

Gradient Accumulation Example

this month medium (<30 mins) | pytorch implementation training gradients

Simulate large-batch training using gradient accumulation: - Train with microbatches of size 4. - Accumulate gradients over 8 steps. - Update optimizer after accumulation. Verify final result...

Implement Early Stopping

this month medium (<1 hr) | pytorch training basics early stopping

Add early stopping to a training loop: - Monitor validation loss. - Stop training if no improvement after 5 epochs. - Save best model checkpoint. Demonstrate on MNIST subset.

Implement Label Smoothing

this month medium (<30 mins) | pytorch training label smoothing classification

Write a function to apply label smoothing for classification: - Replace one-hot targets with $$1-\epsilon$$ for true class, $$\epsilon/(K-1)$$ for others. - Use it in cross-entropy training. Show...

Save and Load TorchScript Model

this month medium (<30 mins) | pytorch torchscript model deployment serialization

Convert a trained PyTorch model to TorchScript via tracing and scripting. Save it to disk. Reload and run inference. Compare outputs with the original model.

Mixed Precision Training with autocast

this month medium (<30 mins) | pytorch training mixed precision gpu

Modify a training loop to use `torch.cuda.amp.autocast`: - Wrap forward + loss in `autocast`. - Use `GradScaler` for backward. Compare training speed vs. full precision.