Improve Fiber::ExecutionContext.default_workers_count#16149
Conversation
This I feel would be a mistake. While from a pure processing point of view it would probably be ok, there are other situations where you do outgoing calls in some way that may block execution but might not be very costly from a CPU perspective. The classical example would be a thread pool having a bunch of threads reading from spinning disks in parallel - then it may make sense to overcommit by a lot. I suppose the usefulness of that particular example goes down when using io_uring as the event loop, but there are other situations where it may make sense. EDIT: Oh, this caps only the DEFAULT value? Ok, that make a lot of sense. Never mind then |
|
It's not even the actual default parallelism, it's the one we recommend when you opt in to MT by resizing the default context. |
- Makes sure there is at least 1 worker (minimum required). - Doesn't cap the total to an arbitrary number. - Caps the total to the effective logical CPUs when supported (linux).
70ddc98 to
f619d2f
Compare
| total = System.cpu_count.to_i.clamp(1..) | ||
| effective = Crystal::System.effective_cpu_count.to_i.clamp(1..) |
There was a problem hiding this comment.
Aren't #to_i calls redundant (since .cpu_count and .effective_cpu_count methods already return an Int)?
| total = System.cpu_count.to_i.clamp(1..) | |
| effective = Crystal::System.effective_cpu_count.to_i.clamp(1..) | |
| total = System.cpu_count.clamp(1..) | |
| effective = Crystal::System.effective_cpu_count.clamp(1..) |
There was a problem hiding this comment.
No. They may return any Int. For example on Linux it's an u64 or i64, but EC expects an i32.
There was a problem hiding this comment.
Gotcha, thanks for the explanation 🙏
| total = System.cpu_count.to_i.clamp(1..) | ||
| effective = Crystal::System.effective_cpu_count.to_i.clamp(1..) | ||
| # TODO: query for CPU limits (e.g. linux/cgroup, freebsd/rctl, ...) | ||
|
|
||
| Math.min(total, effective) |
There was a problem hiding this comment.
@ysbaddaden IIUC if either one of the total or effective is not supported by the OS, the corresponding method call will return -1, which will turn into 1 following the #clamp(1..) call, which then in turn will limit the overall count and method's return value to 1, via Math.min(x, 1).
Is that an expected behavior? 🤔
There was a problem hiding this comment.
Uuurgh, you're right, -1 shall skip the value 😭
Depends on #16148.