All Katas - ML Katas

Advanced Indexing with `gather` for NLP

this month medium (<30 mins) | NLP gather indexing

In Natural Language Processing, it's common to work with sequences of varying lengths. A frequent task is to extract the activations of the last token in each sequence from a tensor of shape...

Implementing a simplified Beam Search Decoder with `gather` and `scatter`

this month hard (>1 hr) | NLP scatter gather

Beam search is a popular decoding algorithm used in machine translation and text generation. A key step in beam search is to select the top-k most likely next tokens and update the corresponding...

Replicating `torch.nn.Embedding` with `gather`

this month medium (<30 mins) | embeddings NLP gather

The `torch.nn.Embedding` layer is fundamental in many deep learning models, especially in NLP. Your task is to replicate its forward pass functionality using `torch.gather`. You'll create a...

Deconstructing Self-Attention Scores

this year hard (>1 hr) | Attention Transformers Softmax NLP

The self-attention mechanism is a core component of Transformers. Let's break down how attention scores are calculated. 1. **Query, Key, Value**: In self-attention, each input token (or its...