I'm a little skeptically of the Swish implementation after looking at Table 2.

Method | Top 1 | Top 5 No-op | 78% | 94% Swish | 64.5% | 86% Mish | 79% | 94.5%

Swish is the only value that decreases performance (and by a huge magnitude) but a very related methodology improves performance hummm...