All Katas - ML Katas

Tiny Neural Radiance Fields (NeRF)

yesterday medium (<30 mins) | pytorch nerf implicit nets

### Description Implement a simplified version of a Neural Radiance Field (NeRF) to represent a 2D image. [1] A NeRF learns a continuous mapping from spatial coordinates to pixel values. Instead...

Implement Lottery Ticket Hypothesis Pruning

yesterday medium (<30 mins) | pytorch pruning lottery ticket

### Description The Lottery Ticket Hypothesis suggests that a randomly initialized, dense network contains a smaller subnetwork (a "winning ticket") that, when trained in isolation, can match the...

Simple Differentiable Renderer

yesterday medium (<30 mins) | pytorch autograd diff rendering 3d

### Description Modern 3D deep learning often relies on differentiable rendering, allowing gradients to flow from a 2D rendered image back to 3D scene parameters. [1] Your task is to implement a...

Spiking Neuron with Leaky Integrate-and-Fire

yesterday hard (<30 mins) | pytorch snn spiking neuroscience

### Description Implement a single Leaky Integrate-and-Fire (LIF) neuron, the fundamental building block of many Spiking Neural Networks (SNNs). Unlike traditional neurons, LIF neurons operate on...

Implementing a Custom Loss Function with `torch.autograd`

this month medium (<30 mins) | regression loss custom function huber

Create a **custom loss function** that inherits from `torch.nn.Module` and performs a non-standard calculation. For example, a custom Huber loss. This loss is less sensitive to outliers than Mean...

Custom `DataLoader` for On-the-Fly Image Generation

this month medium (<30 mins) | dataset dataloader custom generative synthetic

Create a **custom `torch.utils.data.Dataset`** that doesn't load data from disk. Instead, the `__getitem__` method should **generate** an image on the fly (e.g., a simple geometric shape, a random...

Custom Data Augmentation Pipeline

this month medium (<30 mins) | custom data augmentation transforms pipeline

Create a **custom data augmentation pipeline** using PyTorch's `transforms`. For a given dataset (e.g., a custom image dataset), implement a series of transformations like random rotation,...

Building a Custom `Dataset` and `DataLoader`

this month medium (<30 mins) | dataset dataloader custom data pipeline

Create a **custom `torch.utils.data.Dataset` class** to load a simple, non-image dataset (e.g., from a CSV file). The `__init__` method should read the data, `__len__` should return the total...

Implementing Layer Normalization from Scratch

this month medium (<30 mins) | training custom module normalization layer

Implement **Layer Normalization** as a custom `torch.nn.Module`. Unlike `BatchNorm`, `LayerNorm` normalizes across the features of a single sample, not a batch. Your implementation should...

Manual Gradient Descent Step

this month medium (<30 mins) | autograd optimization gradient descent

Simulate one step of gradient descent for a simple quadratic loss. ### Problem Given a scalar parameter $w$ initialized at 5.0, minimize the loss $L(w) = (w - 3)^2$ using PyTorch. - **Input:**...

Custom Dataset Class

this month medium (<30 mins) | dataloader nn datasets

Create a custom PyTorch `Dataset` for pairs of numbers and their sum. ### Problem Implement a dataset where each sample is `(x, y, x+y)`. - **Input:** A list of tuples `(x, y)`. - **Output:** For...

Implement a Simple MLP

this month medium (<30 mins) | mlp nn forward pass

Build and run a minimal Multi-Layer Perceptron (MLP) using `torch.nn`. ### Problem Construct a 2-layer MLP with ReLU activation for input of size 10 and output of size 2. - **Input:** Tensor of...

Implement a Custom Loss Function

this month medium (<30 mins) | pytorch loss functions regression implementation

Create a custom loss function called `MeanAbsolutePercentageError` (MAPE) in PyTorch. It should: 1. Take predictions and targets as input tensors. 2. Compute $$\frac{1}{n} \sum_i \frac{|y_i -...

Gradient Clipping Example

this month medium (<30 mins) | pytorch rnn training gradients

Write code to: 1. Train a small RNN on dummy data. 2. Add gradient clipping using `torch.nn.utils.clip_grad_norm_`. 3. Print gradient norms before and after clipping. Show that exploding gradients...

Implement a Linear Regression Model

this month easy (<30 mins) | pytorch training linear regression basics

Build a simple linear regression model using `nn.Module`. Requirements: - One input feature, one output. - Train it on synthetic data $$y = 3x + 2 + \epsilon$$. - Use `MSELoss` and `SGD`. Check...

Checkpointing with torch.save

this month easy (<30 mins) | pytorch training basics checkpoints

Train a simple feedforward model for 1 epoch. Save: 1. Model state dict. 2. Optimizer state dict. 3. Epoch number. Then load the checkpoint and resume training seamlessly.

Weight Initialization Techniques

this month medium (<30 mins) | pytorch deep learning training initialization

Initialize a neural network's weights using different schemes: - Xavier initialization. - Kaiming initialization. Show histograms of weight distributions before and after initialization.

Custom Collate Function

this month medium (<30 mins) | pytorch dataloader sequences collate

Write a custom `collate_fn` for `DataLoader` that pads variable-length sequences with zeros. Use `torch.nn.utils.rnn.pad_sequence`. Test by batching random-length tensors.

Gradient Accumulation Example

this month medium (<30 mins) | pytorch implementation training gradients

Simulate large-batch training using gradient accumulation: - Train with microbatches of size 4. - Accumulate gradients over 8 steps. - Update optimizer after accumulation. Verify final result...

Implement Label Smoothing

this month medium (<30 mins) | pytorch training label smoothing classification

Write a function to apply label smoothing for classification: - Replace one-hot targets with $$1-\epsilon$$ for true class, $$\epsilon/(K-1)$$ for others. - Use it in cross-entropy training. Show...

Save and Load TorchScript Model

this month medium (<30 mins) | pytorch torchscript model deployment serialization

Convert a trained PyTorch model to TorchScript via tracing and scripting. Save it to disk. Reload and run inference. Compare outputs with the original model.

Mixed Precision Training with autocast

this month medium (<30 mins) | pytorch training mixed precision gpu

Modify a training loop to use `torch.cuda.amp.autocast`: - Wrap forward + loss in `autocast`. - Use `GradScaler` for backward. Compare training speed vs. full precision.