All Katas - ML Katas

Einops: ViT-Style Patch Embedding

yesterday easy (<10 mins) | transformer einops vision tensor

### Description Vision Transformers (ViT) process images by first breaking them down into a sequence of flattened patches. The `einops` library is perfectly suited for this task, offering a...

Tensor Manipulation: Causal Mask for Transformers

yesterday easy (<10 mins) | transformer attention tensor

### Description In decoder-style Transformers (like GPT), we need a "causal" or "look-ahead" mask to prevent positions from attending to subsequent positions. This is typically a lower-triangular...

Tensor Manipulation: Sinusoidal Positional Encoding

yesterday hard (<10 mins) | transformer attention nlp tensor

### Description Implement the sinusoidal positional encoding from the "Attention Is All You Need" paper. This technique adds information about the position of tokens in a sequence by injecting a...

Einops: Transpose for Attention Output

yesterday easy (<10 mins) | transformer attention einops

### Description After the multi-head attention calculation, the output tensor typically has the shape `(B, num_heads, N, head_dim)`. To feed this into the next layer (usually a feed-forward...