Prensent the theory behind SWISH activation function, discuss benefits and limitations. Define a simple classification problem, implement a basline solution and compare results using SWISH with respect to simpler activations like ReLU and ELU.
Searching for Activation Functions
pip3 install torch torchvision torchaudio
pip install datetime