palle-k / dl4s Goto Github PK

Mish is a new novel activation function proposed in this paper.
It has shown promising results so far and has been adopted in several packages including:

• TensorFlow-Addons
• SpaCy (Tok2Vec Layer)
• Thinc - SpaCy's official NLP based ML library
• Echo AI
• CNTKX - Extension of Microsoft's CNTK
• FastAI-Dev
• Darknet
• Yolov3
• BeeDNN - Library in C++
• Gen-EfficientNet-PyTorch
• dnet

All benchmarks, analysis and links to official package implementations can be found in this repository

It would be nice to have Mish as an option within the activation function group.

This is the comparison of Mish with other conventional activation functions in a SEResNet-50 for CIFAR-10:

I just finished a script for changing the image. I couldn't find any utility online that let me just change individual pixels' colors - so frustrating. Anyway, now all you have to do is type in the 8-bit color values and the script will modify the image. Luckily, the edges of letters all had the same RGB, just different alpha.

The script produces the same output as it got as input by default. I'm giving this to you in case you want to experiment with the color scheme yourself, but I'll experiment myself and see what looks good.

main.swift.zip

Hey there @palle-k, I really love this project and I think it is really awesome! I'd really like to help out and contribute, but DL4S seems pretty complete to me. Is there anything that I could help with? I'm really familiar with Swift, and I'm a bit new to open source.

Love this project, as well as your Seq2Seq implementation and RL library!

Have you considered integrating with Swift for TensorFlow? 😄

Use this thread from now on for continued discussion about DL4S