All Katas - ML Katas

Implementing a Multi-Headed Attention Mechanism

this month hard (>1 hr) | transformer attention scratch nlp multi-head

Expand on the previous attention exercise by implementing a **Multi-Headed Attention mechanism** from scratch. A single attention head is a dot-product attention as you've implemented. Multi-head...

Implementing a Masked Language Model

this month hard (>1 hr) | nlp model masked language bert

Implement a **Masked Language Model (MLM)**, a technique at the heart of models like BERT. Given a sentence, you'll randomly mask some of the words and then train a model to predict those masked...

Implementing a Simple VAE for Text (Sentence VAE)

this month hard (>1 hr) | rnn vae generative text nlp

Implement a **Variational Autoencoder (VAE)** for text, often called a Sentence VAE. The encoder will be an RNN (e.g., GRU) that outputs a latent distribution, and the decoder will be another RNN...