Feature Request: Add Mish activation

[digantamisra98]

Mish is a new novel activation function proposed in this paper.
It has shown promising results so far and has been adopted in several packages including:

• TensorFlow-Addons
• SpaCy (Tok2Vec Layer)
• Thinc - SpaCy's official NLP based ML library
• Echo AI
• Eclipse's deeplearning4j
• Hasktorch
• CNTKX - Extension of Microsoft's CNTK
• FastAI-Dev
• Darknet
• Yolov3
• BeeDNN - Library in C++
• Gen-EfficientNet-PyTorch
• dnet

All benchmarks, analysis and links to official package implementations can be found in this repository

It would be nice to have Mish as an option within the activation function group.

This is the comparison of Mish with other conventional activation functions in a SEResNet-50 for CIFAR-10: (Better accuracy and faster than GELU)

[JamesLear92]

I've just raised a pull request to implement this:
#94

[digantamisra98]

@JamesLear92 Thanks for the consideration

[JamesLear92]

@digantamisra98
I've now got it working on cuda so the pull request is ready for merge

[digantamisra98]

@JamesLear92 awesome. I'll look forward to giving it a try once merged. Thanks!

[JamesLear92]

@Sergio0694
Hi, are you happy to merge the my pull request?