All Katas - ML Katas

by: newest upvotes saves

Implement a Knowledge Distillation Loss

this year medium (<30 mins) | pytorch compression distillation

### Description Knowledge Distillation is a model compression technique where a small "student" model is trained to mimic a larger, pre-trained "teacher" model. [1] This is achieved by training...
Model Compression with Pruning

this year medium (>1 hr) | training pruning model compression sparsify

Implement **model pruning** to reduce the size and computational cost of a trained model. Start with a simple, over-parameterized model (e.g., a fully-connected network on MNIST). Train it to a...