Hello,
std::thread::hardware_concurrency() returns, when possible, the underlying hardware capability to run threads, which might not corresponds to the actual number of cores available to the process (through the use of taskset, batch system like slurm, etc...). The consequence is that mkdwarfs might run in a non optimal way. For example, if I run taskset -c 1 mkdwarfs on my 20 cores machines, it will run 20 workers on only one core.
The immediate workaround is to use the -N option to set the number of workers, but I think a more sane behavior would be to use sched_getaffinity as in opencv/opencv#16268. Gromacs did something similar (https://github.com/gromacs/gromacs/blob/1e6873fadf16d5f5be861e6f9ef5f9923a12e540/src/gromacs/hardware/hardwaretopology.cpp#L1221).
What do you think ?
Thank you.
Hello,
std::thread::hardware_concurrency()returns, when possible, the underlying hardware capability to run threads, which might not corresponds to the actual number of cores available to the process (through the use of taskset, batch system like slurm, etc...). The consequence is thatmkdwarfsmight run in a non optimal way. For example, if I runtaskset -c 1 mkdwarfson my 20 cores machines, it will run 20 workers on only one core.The immediate workaround is to use the -N option to set the number of workers, but I think a more sane behavior would be to use sched_getaffinity as in opencv/opencv#16268. Gromacs did something similar (https://github.com/gromacs/gromacs/blob/1e6873fadf16d5f5be861e6f9ef5f9923a12e540/src/gromacs/hardware/hardwaretopology.cpp#L1221).
What do you think ?
Thank you.