All Katas - ML Katas

by: newest upvotes saves

Implementing a simplified Beam Search Decoder with `gather` and `scatter`

this month hard (>1 hr) | NLP scatter gather

Beam search is a popular decoding algorithm used in machine translation and text generation. A key step in beam search is to select the top-k most likely next tokens and update the corresponding...
Deconstructing Self-Attention Scores

this year hard (>1 hr) | Attention Transformers Softmax NLP

The self-attention mechanism is a core component of Transformers. Let's break down how attention scores are calculated. 1. **Query, Key, Value**: In self-attention, each input token (or its...