I think the threshold of 20.0f in std::min can be tweaked to improve accuracy. It mostly won't make any difference though. But for the record, here are graphs:

The graphs compare three implementations: x * tanh(log1p(exp(x))), x * tanh(log(1 + exp(x))) and another for which source is given above the graph.

Current:

float eX = exp(std::min(x, 20.f));
float n = (eX + 2) * eX;
dstptr[i] = (x * n) / (n + 2);

New:

if (x > 8.688687f)
    return x;
else
{
    float eX = std::exp(x);
    float n = (eX + 2) * eX;
    return (x * n) / (n + 2);
}

If we assume that the distribution of activation inputs is normally distributed and centred at zero, we can drop a hint to the compiler that x > 8.688687f path is rare.

@YashasSamaga, thanks! So somebody new to open a PR :) Feel free to propose a patch or, if you don't mind, we can ask one of our local students to reproduce this experiment and make their first open source contribution 😃

we can ask one of our local students to reproduce this experiment and make their first open source contribution

This sounds better.

And here is the code I used to generate data and plots if it helps:

#include <cmath>
#include <cstdio>
#include <algorithm>

float mish2(float x)
{
    if (x > 8.688687f)
        return x;
    else
    {
        float eX = exp(std::min(x, 20.f));
        float n = (eX + 2) * eX;
        return (x * n) / (n + 2);
    }
}

int main ()
{
    for (float x = -30; x < 30; x += 0.0001)
    {
        //float activation = x * std::tanh(std::log(1 + std::exp(x)));
        //float activation = x * std::tanh(std::log1p(std::exp(x)));
        float activation = mish2(x);
        std::printf("%.7f %.7e\n", x, activation);
    }
    return 0;
}

Plot code: https://github.com/YashasSamaga/ConvolutionBuildingBlocks/blob/master/mish/plot.py

Reference values are computed directly by the formula with 128-bit floats.