ML Katas

Building on the previous exercise, let's switch to Triplet Loss. This loss function is more powerful as it enforces a margin between an anchor-positive pair and an anchor-negative pair. The loss is defined as:

Where $d$ is the distance function (e.g., Euclidean distance), $a$ is the anchor, $p$ is the positive sample, $n$ is the negative sample, and $\alpha$ is a margin. The key challenge here is to create batches of triplets $(a,p,n)$ from your dataset. A simple but effective method is to create positive pairs from the same class and negative pairs from different classes within a batch.

Verification: After training, the model should produce embeddings where the distance between an anchor and a positive sample is consistently smaller than the distance between the anchor and a negative sample, by at least the margin $\alpha$. You can verify this by calculating and plotting the distances on a test set.