ML Katas

Custom Activation Function

medium (<1 hr) pytorch implementation activations mlp
this year by E

Define a custom activation function called Swish: f(x)=x·σ(x).

  • Implement it as a PyTorch nn.Module.
  • Train a small MLP on random data with it.
  • Compare with ReLU performance (loss curves).