We run a CI product (Bitbucket Pipelines) and have had several reports from customers about degraded performance when they upgraded their node.js based builds to node 22.12 or above. Part of this upgrade involves a change to libuv. Our infrastructure is based on Kubernetes and we run with cgroups v2. From our investigation, it seems theres a change in behaviour around how parallelism is calculated. The impacted libuv version is 1.49.1, node 22.12, affecting Linux x86 and ARM.

The problematic function looks to be node.js require('os').availableParallelism(), which calls into libuv uv_available_parallelism. Depending on the cgroups implementation or libuv library version used, we get different results for available parallelism, which can go on to impact node.js “performance”, where programs may run with less parallelism.=

We’ve run experiments, looking at the result of os.availableParallelism() as well as inspecting cgroup files that are involved in calculating parallelism.

In all tests, require('os').cpus().length evaluates to 4 and we've assigned 4 cores to the pod (we are running inside of light weight VMs - this is the equivalent of running on a host with 4 cores). The same behaviour has been observed both with and without lightweight VMs wrapping pods, and we've narrowed down to a cgroups v1 vs v2 different in behaviour with libuv. We've seen this happen when upgrading from node 22.11 to 22.12 and above.

We've noticed that where a container has no requests set against it, libuv 1.49.1 returns a different result when running under cgroups v1 vs v2. Despite a container being configured to have a limit of 4 cores, libuv returns a parallelism of 1 under cgroups v2. Under cgroups v1, we get a parallelism of 4.

Expected Behaviour

When swapping from cgroups v1 to cgroups 2, uv_available_parallelism returns the same value.

Actual Behaviour

uv_available_parallelism reduces a reduced value when moving to cgroups v2.

Experiments

Ive run several experiments in Kubernetes with various pod configs, and have recorded the difference in parallelism for each container in the pod when under cgroups v1 vs v2, and also attached information from /sys/fs/cgroup related to cpus that is used by uv_available_parallelism calculations.

Single container pod with no requests:

   - image: <TestImage>
      imagePullPolicy: Always
      name: container-1
      resources:
        limits:
          cpu: "4"
          memory: 2Gi
        requests:
          cpu: "0"
          memory: 2Gi

Result

cgroups v1 available parallelism: 4
cgroups v2 available parallelism: 1 - incorrect, parallelism now reduced

cgroup cpu details

container-1 details under cgroups v1:

cpu.shares: 2
cpu.cfs_quota_us: 400000
cpu.cfs_period_us: 100000

container-1 details under cgroups v2:

cpu.weight: 1
cpu.max: 400000 100000

Single container pod with requests:

    - image: <TestImage>
      imagePullPolicy: Always
      name: container-1
      resources:
        limits:
          cpu: "4"
          memory: 2Gi
        requests:
          cpu: "4"
          memory: 2Gi

Result

cgroups v1 available parallelism: 4
cgroups v2 available parallelism: 4 - incorrect, parallelism now reduced

cgroup cpu details

container-1 detals under cgroups v1:

cpu.shares: 4096
cpu.cfs_quota_us: 400000
cpu.cfs_period_us: 100000

container-1 details under cgroups v2:

cpu.weight: 157
cpu.max: 400000 100000

Multi container pod:

    - image: <TestImage>
      imagePullPolicy: Always
      name: container-1
      resources:
        limits:
          cpu: "4"
          memory: 2Gi
        requests:
          cpu: "4"
          memory: 2Gi
    - image: <TestImage>
      imagePullPolicy: Always
      name: container-2
      resources:
        limits:
          cpu: "4"
          memory: 2Gi
        requests:
          cpu: "0"
          memory: 2Gi

Result

cgroups v1 container1: available parallelism: 4
cgroups v1 container2: available parallelism: 4

cgroups v2 container1: available parallelism: 4
cgroups v2 container2: available parallelism: 1 - incorrect, parallelism now reduced

cgroup cpu details

container1 cgroups v1:

cpu.shares: 4096
cpu.cfs_quota_us: 400000
cpu.cfs_period_us: 100000

container1 cgroups v2:

cpu.weight: 157
cpu.max: 400000 100000

container2 cgroups v1

cpu.shares: 2
cpu.cfs_quota_us: 400000
cpu.cfs_period_us: 100000

container2 cgroups v2

cpu.weight: 1
cpu.max: 400000 100000