The normalization may not be needed, std::discrete_distribution already divides by the sum.

I can confirm that normalization is not needed.

Tested with following program, output is the same for {10, 10, 10, 100} and {100, 100, 100, 1000}

#include <iostream>
#include <iomanip>
#include <map>
#include <random>
 
int main()
{
    std::random_device rd;
    std::mt19937 gen(rd());
    // std::discrete_distribution<> d({10, 10, 10, 100});
    std::discrete_distribution<> d({100, 100, 100, 1000});
    std::map<int, int> map;
 
    for (int n = 0; n < 1e4; ++n)
        ++map[d(gen)];
 
    for(const auto& [num, count] : map)
        std::cout << num << " generated " << std::setw(4) << count << " times\n";
}

On a related note, I tried an optimization (which is getting out of scope for this PR): during the initial pass over logits and the application of temperature scaling, there's a way to bypass tokens that have probabilities much lower than the highest probability using a heuristic:

const float maxl = *std::max_element(plogits, plogits + n_logits);
const float minp_ratio = 1e-6; // Ratio lowest admissible prob : highest prob
const float minl = maxl / repeat_penalty + log(minp_ratio) * temp; // Assumes the best token is penalized
// Then, when plogits[i] < minl, don't push_back the pair(logit, i) into logits_id

Sampling is clearly not the most compute heavy task, so it might not be a good idea to introduce a heuristic that could potentially fail. I was experimenting with a more complex repetition search, that could justify it.