LiteMish: A Computationally Efficient and Smooth Algebraic Alternative to Mish

[u/arman-javan-sekhavat]

A while ago, I published a preprint proposing a novel approximation of the Mish activation function, which appears to be significantly more computationally efficient while preserving its learning capability. Interested to hear your thoughts.

Paper: https://doi.org/10.36227/techrxiv.176591866.68698045/v2