Skip to content

Add inject_gomaxprocs config (dynamic calculation approach)#9844

Closed
harche wants to merge 1 commit into
cri-o:mainfrom
harche:inject-gomaxprocs-dynamic
Closed

Add inject_gomaxprocs config (dynamic calculation approach)#9844
harche wants to merge 1 commit into
cri-o:mainfrom
harche:inject-gomaxprocs-dynamic

Conversation

@harche

@harche harche commented Mar 25, 2026

Copy link
Copy Markdown
Contributor

Adds a new CRI-O config option inject_gomaxprocs that injects the
GOMAXPROCS environment variable into burstable and best-effort pod
containers. For burstable pods, the value is auto-calculated from
the CPU request (via cpu_shares). The configured value acts as a
minimum floor. Guaranteed pods, workload-partitioned pods, and pods
with GOMAXPROCS already set are skipped.

Also adds a skip-gomaxprocs.crio.io pod annotation for per-pod opt-out.

Jira: OCPNODE-4221
Fixes: OCPBUGS-64621 — Containerized OpenShift components showing significant CPU spikes on systems with high core count

Summary by CodeRabbit

  • New Features

    • New global option/runtime config --inject-gomaxprocs to inject GOMAXPROCS into container environments (default 0 disables). Burstable pods auto-calc from CPU requests with a configurable minimum floor; best-effort pods use the configured fallback. Skips guaranteed and workload-partitioned pods; per-pod opt-out via skip-gomaxprocs.crio.io annotation.
  • Documentation

    • CLI help, man page, config docs and shell completions (bash, fish, zsh) updated for the new option.
  • Tests

    • Added unit and integration tests validating injection behavior, calculations, and opt-outs.
Add inject_gomaxprocs config option to dynamically set GOMAXPROCS for burstable and best-effort pod containers based on CPU request, with a configurable minimum floor.

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 25, 2026
@openshift-ci

openshift-ci Bot commented Mar 25, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci Bot added the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label Mar 25, 2026
@coderabbitai

coderabbitai Bot commented Mar 25, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds configurable GOMAXPROCS injection: new runtime config and CLI flag, per-pod opt-out annotation, workload-partition detection, CPU-share-based calculation and conditional injection during container creation, plus unit and integration tests and updated shell completions and docs.

Changes

Cohort / File(s) Summary
Shell completions
completions/bash/crio, completions/fish/crio.fish, completions/zsh/_crio
Added --inject-gomaxprocs completion entries; Fish completion accepts a value and includes help text.
CLI flag handling
internal/criocli/criocli.go
Registered --inject-gomaxprocs as an Int64Flag (env CONTAINER_INJECT_GOMAXPROCS) and merged explicit CLI value into runtime config.
Runtime configuration
pkg/config/config.go, pkg/config/template.go
Added InjectGOMAXPROCS int64 to RuntimeConfig (TOML inject_gomaxprocs), validation to reject negative values, and template emission for non-default values.
Annotations
pkg/annotations/v2/annotations.go
Added SkipGoMaxProcs = "skip-gomaxprocs.crio.io" and included it in exported AllAnnotations.
Workload partition detection
pkg/config/workloads.go
Added Workloads.IsWorkloadPartitioned(sboxAnnotations map[string]string) bool helper to detect workload-partitioned pods.
Injection implementation
server/container_create.go
Passed sb into env/workdir setup, added calculateGOMAXPROCS and injectGOMAXPROCS, and implemented conditional injection: honor opt-out annotation, skip workload-partitioned and guaranteed pods, detect burstable/besteffort from cgroup parent, skip when CPU quota exists, and avoid injecting if GOMAXPROCS already present.
Unit & integration tests
server/container_create_gomaxprocs_test.go, test/inject_gomaxprocs.bats
Added unit tests for calculation, injection, and workload partition detection; Bats integration tests covering burstable/besteffort/guaranteed cases, opt-out, explicit values, CPU-limit skip, and disabling via config/env.
Docs & manpages
docs/crio.8.md, docs/crio.conf.5.md
Documented new global CLI option --inject-gomaxprocs and crio.runtime.inject_gomaxprocs config option, behavior, and disable semantics (default 0).

Sequence Diagram

sequenceDiagram
    participant Client
    participant Server as ContainerCreate
    participant Config as RuntimeConfig
    participant Sandbox as Sandbox
    participant Spec as OCI_Spec

    Client->>Server: Create container request
    Server->>Config: Read InjectGOMAXPROCS
    alt Inject disabled (<=0)
        Server->>Spec: Proceed without injection
    else Inject enabled (>0)
        Server->>Sandbox: Fetch annotations & cgroup parent
        Sandbox-->>Server: Return annotations, cgroup info
        Server->>Server: Check skip-gomaxprocs annotation / workload partition
        alt Skip present or workload-partitioned
            Server->>Spec: Skip injection
        else
            Server->>Server: Determine burstable vs besteffort from cgroup parent
            Server->>Server: calculateGOMAXPROCS(cpuShares, fallback)
            Server->>Server: Check existing GOMAXPROCS in default/merged env
            alt GOMAXPROCS already set
                Server->>Spec: Preserve existing value
            else
                Server->>Spec: injectGOMAXPROCS(value)
            end
        end
    end
    Spec-->>Client: Container ready
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Suggested labels

kind/feature

Suggested reviewers

  • hasan4791
  • mrunalp

Poem

🐰 I hop through flags and docs with glee,
I count CPU shares to set GOMAXPROCS for thee.
Skip me with an annotation or set me to 0,
I floor and fallback so programs can grow.
A tiny rabbit patch—swift, soft, and merry.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 27.27% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The PR title accurately describes the main feature: adding inject_gomaxprocs config with a dynamic calculation approach. It directly reflects the primary changeset focused on this new configuration option.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot added the dco-signoff: yes Indicates the PR's author has DCO signed all their commits. label Mar 25, 2026
@openshift-ci

openshift-ci Bot commented Mar 25, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: harche
Once this PR has been reviewed and has the lgtm label, please assign haircommander for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@codecov

codecov Bot commented Mar 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 51.56250% with 31 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.16%. Comparing base (848f93a) to head (49a75db).
⚠️ Report is 2 commits behind head on main.

❗ There is a different number of reports uploaded between BASE (848f93a) and HEAD (49a75db). Click for more details.

HEAD has 15 uploads less than BASE
Flag BASE (848f93a) HEAD (49a75db)
22 7
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #9844       +/-   ##
===========================================
- Coverage   67.61%   56.16%   -11.45%     
===========================================
  Files         212      212               
  Lines       29366    29428       +62     
===========================================
- Hits        19856    16529     -3327     
- Misses       7814    11291     +3477     
+ Partials     1696     1608       -88     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@harche harche force-pushed the inject-gomaxprocs-dynamic branch from f046fbc to 905f1bf Compare March 26, 2026 17:03
@rphillips

Copy link
Copy Markdown
Contributor

I believe we will want an annotation to skip injecting GOMAXPROCS for a specific pod, unless it already exists via the pod yaml.

Comment thread server/container_create.go Outdated
}
}

log.Infof(ctx, "Injecting GOMAXPROCS=%d", maxProcs)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably in a hot path. We are not logging the name of the pod either. Perhaps delete this line?

@harche harche force-pushed the inject-gomaxprocs-dynamic branch 3 times, most recently from 6e475f7 to af1fa10 Compare March 26, 2026 21:43
@openshift-ci openshift-ci Bot added dco-signoff: no Indicates the PR's author has not DCO signed all their commits. and removed dco-signoff: yes Indicates the PR's author has DCO signed all their commits. labels Mar 26, 2026
@harche harche force-pushed the inject-gomaxprocs-dynamic branch from af1fa10 to 8a56d93 Compare March 26, 2026 21:45
@openshift-ci openshift-ci Bot added dco-signoff: yes Indicates the PR's author has DCO signed all their commits. and removed dco-signoff: no Indicates the PR's author has not DCO signed all their commits. labels Mar 26, 2026
@harche harche marked this pull request as ready for review March 26, 2026 21:45
@harche harche requested a review from mrunalp as a code owner March 26, 2026 21:45
@openshift-ci openshift-ci Bot requested review from QiWang19 and hasan4791 March 26, 2026 21:45
@harche harche force-pushed the inject-gomaxprocs-dynamic branch from 8a56d93 to c3d598e Compare March 26, 2026 21:49

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/config/config.go`:
- Around line 401-410: Add a runtime validation that rejects negative values for
InjectGOMAXPROCS: in the central config validation function (where other config
fields are validated) check that InjectGOMAXPROCS >= 0 and return a clear error
(e.g., "inject_gomaxprocs must be >= 0") if it is negative so misconfiguration
is surfaced at startup rather than silently treated as disabled.

In `@test/inject_gomaxprocs.bats`:
- Around line 17-22: The fixtures in test/inject_gomaxprocs.bats use capitalized
QoS names which don't match the lowercase checks in
setupContainerEnvironmentAndWorkdir and sb.CgroupParent(); update the jq
assignments to set .linux.cgroup_parent to lowercase "burstable-pod_test.slice"
and "besteffort-pod_test.slice" (apply same change to the other
burstable/best-effort fixtures referenced) so the auto-injection and
preservation branches are exercised, and also fix the cpu_shares comment/value
mismatch by making the comment and the jq-ed value consistent (either change the
comment to 8192 or set .linux.resources.cpu_shares to 2048) so the fixture
intent matches its data.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 8f69f363-de58-4d39-9e6e-109c1cf9cc09

📥 Commits

Reviewing files that changed from the base of the PR and between 848f93a and 8a56d93.

📒 Files selected for processing (12)
  • completions/bash/crio
  • completions/fish/crio.fish
  • completions/zsh/_crio
  • docs/crio.8.md
  • internal/criocli/criocli.go
  • pkg/annotations/v2/annotations.go
  • pkg/config/config.go
  • pkg/config/template.go
  • pkg/config/workloads.go
  • server/container_create.go
  • server/container_create_gomaxprocs_test.go
  • test/inject_gomaxprocs.bats

Comment thread pkg/config/config.go
Comment thread test/inject_gomaxprocs.bats Outdated
@harche harche force-pushed the inject-gomaxprocs-dynamic branch from c3d598e to bf4a504 Compare March 26, 2026 22:05

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
server/container_create.go (1)

1031-1057: Consider removing or reducing verbosity of skip logs in the container creation hot path.

These log.Infof calls (lines 1041, 1043) execute during every container creation. For skip conditions that may fire frequently (e.g., all guaranteed pods, all workload-partitioned pods), this could add noticeable logging overhead. Additionally, the log messages don't include the pod/container name for debugging context.

Consider either removing these logs, changing them to log.Debugf, or adding container/pod identifiers.

♻️ Suggested change: Use Debugf with container context
 		if skipAnnotation == "true" {
-			log.Infof(ctx, "Skipping GOMAXPROCS injection: %s annotation is set", v2.SkipGoMaxProcs)
+			log.Debugf(ctx, "Skipping GOMAXPROCS injection for container %s: %s annotation is set", containerConfig.GetMetadata().GetName(), v2.SkipGoMaxProcs)
 		} else if s.config.Workloads.IsWorkloadPartitioned(annotations) {
-			log.Infof(ctx, "Skipping GOMAXPROCS injection: pod is workload-partitioned")
+			log.Debugf(ctx, "Skipping GOMAXPROCS injection for container %s: pod is workload-partitioned", containerConfig.GetMetadata().GetName())
 		} else if cgroupParent := sb.CgroupParent(); strings.Contains(cgroupParent, "burstable") || strings.Contains(cgroupParent, "besteffort") {
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@server/container_create.go` around lines 1031 - 1057, The two log.Infof calls
that run on every container create (the GOMAXPROCS skip paths) should be made
lower-verbosity and include container context: change them to log.Debugf (or
remove) and add identifying fields such as the sandbox/container name and ID
from the sandbox object used here (e.g., use sb.Name() and sb.ID() or
equivalent) so the messages are only emitted at debug level and contain
pod/container context; update the messages in the branch checks that reference
v2.SkipGoMaxProcs and s.config.Workloads.IsWorkloadPartitioned accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@server/container_create.go`:
- Around line 1031-1057: The two log.Infof calls that run on every container
create (the GOMAXPROCS skip paths) should be made lower-verbosity and include
container context: change them to log.Debugf (or remove) and add identifying
fields such as the sandbox/container name and ID from the sandbox object used
here (e.g., use sb.Name() and sb.ID() or equivalent) so the messages are only
emitted at debug level and contain pod/container context; update the messages in
the branch checks that reference v2.SkipGoMaxProcs and
s.config.Workloads.IsWorkloadPartitioned accordingly.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 8a74502d-c594-48ab-a211-955c58dcc904

📥 Commits

Reviewing files that changed from the base of the PR and between 8a56d93 and c3d598e.

📒 Files selected for processing (13)
  • completions/bash/crio
  • completions/fish/crio.fish
  • completions/zsh/_crio
  • docs/crio.8.md
  • docs/crio.conf.5.md
  • internal/criocli/criocli.go
  • pkg/annotations/v2/annotations.go
  • pkg/config/config.go
  • pkg/config/template.go
  • pkg/config/workloads.go
  • server/container_create.go
  • server/container_create_gomaxprocs_test.go
  • test/inject_gomaxprocs.bats
✅ Files skipped from review due to trivial changes (5)
  • completions/bash/crio
  • docs/crio.8.md
  • docs/crio.conf.5.md
  • test/inject_gomaxprocs.bats
  • server/container_create_gomaxprocs_test.go
🚧 Files skipped from review as they are similar to previous changes (6)
  • pkg/config/workloads.go
  • completions/fish/crio.fish
  • completions/zsh/_crio
  • pkg/config/config.go
  • pkg/annotations/v2/annotations.go
  • pkg/config/template.go

@harche harche changed the title WIP: Add inject_gomaxprocs config (dynamic calculation approach) Add inject_gomaxprocs config (dynamic calculation approach) Mar 26, 2026
@openshift-ci openshift-ci Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. labels Mar 26, 2026
@openshift-ci openshift-ci Bot removed the do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. label Mar 26, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
server/container_create.go (1)

1044-1055: Avoid keying pod QoS detection off cgroupParent text.

sb.CgroupParent() is just the raw cgroup-manager output, so the strings.Contains(..., "burstable") / "besteffort" gate bakes path formatting into the feature. This will be much easier to maintain if the QoS decision comes from stable pod data, or at least from a dedicated helper that owns the cgroup-path parsing.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@server/container_create.go` around lines 1044 - 1055, The QoS detection
should not rely on raw cgroup path string checks on sb.CgroupParent(); replace
the strings.Contains(...) gate with a stable source or a dedicated parser:
obtain the pod QoS from the pod metadata (or add a small helper like
getPodQoSFromCgroup(cgroupPath string) that centralizes and tests path parsing)
and base the branch on that QoS value before calling calculateGOMAXPROCS and
injectGOMAXPROCS; update the conditional around sb.CgroupParent(), use the
helper or pod-level QoS field, and keep calculateGOMAXPROCS(specgen, ...) and
injectGOMAXPROCS(specgen, envs, s.ContainerServer.Config().DefaultEnv, maxProcs)
calls unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@internal/criocli/criocli.go`:
- Around line 1338-1343: The flag definition for the cli.Int64Flag named
"inject-gomaxprocs" currently only documents the best-effort fallback, but the
value is also used as a minimum floor for burstable pods; update the Usage
string on the "inject-gomaxprocs" flag (cli.Int64Flag) to explicitly state that
the value is used as both the fallback for best-effort pods and as the minimum
floor for burstable pods, and note that setting 0 disables injection so users
don't misconfigure it.

---

Nitpick comments:
In `@server/container_create.go`:
- Around line 1044-1055: The QoS detection should not rely on raw cgroup path
string checks on sb.CgroupParent(); replace the strings.Contains(...) gate with
a stable source or a dedicated parser: obtain the pod QoS from the pod metadata
(or add a small helper like getPodQoSFromCgroup(cgroupPath string) that
centralizes and tests path parsing) and base the branch on that QoS value before
calling calculateGOMAXPROCS and injectGOMAXPROCS; update the conditional around
sb.CgroupParent(), use the helper or pod-level QoS field, and keep
calculateGOMAXPROCS(specgen, ...) and injectGOMAXPROCS(specgen, envs,
s.ContainerServer.Config().DefaultEnv, maxProcs) calls unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9c3d2e37-8872-45a1-aa96-db34532a4cfd

📥 Commits

Reviewing files that changed from the base of the PR and between c3d598e and bf4a504.

📒 Files selected for processing (13)
  • completions/bash/crio
  • completions/fish/crio.fish
  • completions/zsh/_crio
  • docs/crio.8.md
  • docs/crio.conf.5.md
  • internal/criocli/criocli.go
  • pkg/annotations/v2/annotations.go
  • pkg/config/config.go
  • pkg/config/template.go
  • pkg/config/workloads.go
  • server/container_create.go
  • server/container_create_gomaxprocs_test.go
  • test/inject_gomaxprocs.bats
✅ Files skipped from review due to trivial changes (7)
  • completions/fish/crio.fish
  • completions/zsh/_crio
  • completions/bash/crio
  • docs/crio.8.md
  • docs/crio.conf.5.md
  • server/container_create_gomaxprocs_test.go
  • pkg/config/config.go
🚧 Files skipped from review as they are similar to previous changes (4)
  • pkg/config/workloads.go
  • pkg/annotations/v2/annotations.go
  • pkg/config/template.go
  • test/inject_gomaxprocs.bats

Comment thread internal/criocli/criocli.go
log.Infof(ctx, "Skipping GOMAXPROCS injection: %s annotation is set", v2.SkipGoMaxProcs)
} else if s.config.Workloads.IsWorkloadPartitioned(annotations) {
log.Infof(ctx, "Skipping GOMAXPROCS injection: pod is workload-partitioned")
} else if cgroupParent := sb.CgroupParent(); strings.Contains(cgroupParent, "burstable") || strings.Contains(cgroupParent, "besteffort") {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more edge case is when cpu limit == cpu request but memory limit != memory request. In that case the pod won't  get exclusive cpus via cpu manager static policy and it will go to burstable slice. For Go > 1.25, it looks at cpu limit to adjust GOMAXPROCS. We should check for cpu limit being set and skip in that scenario for burstable pods.

@harche harche force-pushed the inject-gomaxprocs-dynamic branch 2 times, most recently from 4efa766 to 474093e Compare March 27, 2026 00:06

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
internal/criocli/criocli.go (1)

1338-1343: ⚠️ Potential issue | 🟡 Minor

Mention the burstable floor in the flag help.

The runtime also uses this value as the minimum floor for burstable pods, so the current help text under-describes the behavior.

✏️ Suggested wording
 		&cli.Int64Flag{
 			Name:    "inject-gomaxprocs",
 			Value:   defConf.InjectGOMAXPROCS,
-			Usage:   "Enable GOMAXPROCS injection. Burstable pods auto-calculate from CPU request. This value is the fallback for best-effort pods. 0 to disable.",
+			Usage:   "Enable GOMAXPROCS injection. Burstable pods auto-calculate from CPU request and use this value as a minimum floor; best-effort pods use it directly. 0 to disable.",
 			EnvVars: []string{"CONTAINER_INJECT_GOMAXPROCS"},
 		},
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/criocli/criocli.go` around lines 1338 - 1343, Update the help/usage
string for the cli.Int64Flag with Name "inject-gomaxprocs" (currently using
defConf.InjectGOMAXPROCS) to mention that this value is also used as the minimum
floor for burstable pods (in addition to being the fallback for best-effort
pods), so users understand it affects burstable pod calculation; edit the Usage
text in the cli.Int64Flag block to clearly state both behaviors and how to
disable (0).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@server/container_create.go`:
- Around line 1044-1053: Reshape the nested ifs in the cgroup check to be more
lint-friendly: combine the cgroupParent retrieval and contains checks into one
if (if cgroupParent := sb.CgroupParent(); strings.Contains(...) ||
strings.Contains(...)) and then inline the linux/resources checks into that
block, declaring shares and cpuQuota right where you set them from
linux.GetResources() (use a single if resources := linux.GetResources();
resources != nil { shares = resources.GetCpuShares(); cpuQuota =
resources.GetCpuQuota() }). This removes unnecessary blank lines and nesting
around sb.CgroupParent(), linux, and resources and should satisfy
golangci-lint/verify while keeping the logic for sb.CgroupParent(), linux,
resources.GetCpuShares, and resources.GetCpuQuota intact.

---

Duplicate comments:
In `@internal/criocli/criocli.go`:
- Around line 1338-1343: Update the help/usage string for the cli.Int64Flag with
Name "inject-gomaxprocs" (currently using defConf.InjectGOMAXPROCS) to mention
that this value is also used as the minimum floor for burstable pods (in
addition to being the fallback for best-effort pods), so users understand it
affects burstable pod calculation; edit the Usage text in the cli.Int64Flag
block to clearly state both behaviors and how to disable (0).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: ca2a3a52-8152-437b-9e60-4e2ec17e88e7

📥 Commits

Reviewing files that changed from the base of the PR and between bf4a504 and 4efa766.

📒 Files selected for processing (13)
  • completions/bash/crio
  • completions/fish/crio.fish
  • completions/zsh/_crio
  • docs/crio.8.md
  • docs/crio.conf.5.md
  • internal/criocli/criocli.go
  • pkg/annotations/v2/annotations.go
  • pkg/config/config.go
  • pkg/config/template.go
  • pkg/config/workloads.go
  • server/container_create.go
  • server/container_create_gomaxprocs_test.go
  • test/inject_gomaxprocs.bats
✅ Files skipped from review due to trivial changes (6)
  • completions/bash/crio
  • completions/fish/crio.fish
  • completions/zsh/_crio
  • docs/crio.conf.5.md
  • docs/crio.8.md
  • server/container_create_gomaxprocs_test.go
🚧 Files skipped from review as they are similar to previous changes (4)
  • pkg/config/workloads.go
  • pkg/config/config.go
  • test/inject_gomaxprocs.bats
  • pkg/config/template.go

Comment thread server/container_create.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
internal/criocli/criocli.go (1)

1338-1341: ⚠️ Potential issue | 🟡 Minor

Document the burstable floor behavior here too.

The runtime uses this value as the minimum floor for burstable pods as well, not only as the best-effort fallback. Leaving the old wording here also keeps the generated man page text wrong.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@internal/criocli/criocli.go` around lines 1338 - 1341, The flag definition
for cli.Int64Flag named "inject-gomaxprocs" (using defConf.InjectGOMAXPROCS) has
outdated usage text; update the Usage string to state that this value is both
the fallback for best-effort pods and the minimum floor applied to burstable
pods (and that 0 disables injection). Edit the Usage in the cli.Int64Flag for
"inject-gomaxprocs" to mention "minimum floor for burstable pods" alongside the
best-effort fallback and keep the "0 to disable" note.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/config/workloads.go`:
- Around line 171-172: The IsWorkloadPartitioned method currently treats any
activation matched by workloadGivenActivationAnnotation as "partitioned", which
incorrectly flags non-partitioning overrides (e.g., cpuquota/cpulimit or
experimental annotations in WorkloadConfig) and suppresses GOMAXPROCS injection;
update IsWorkloadPartitioned to check the real partitioning signal instead of
the broad helper—either inspect the returned WorkloadConfig for an explicit
partitioning field/property (e.g., a Partition or IsPartition boolean) or check
for the specific partition annotation key used in your annotations map; if the
broader behavior was intended, rename workloadGivenActivationAnnotation to
reflect that it matches any activation (e.g., workloadGivenActivationOrOverride)
and add a new helper (e.g., workloadIndicatesPartitioning) that only detects the
partitioning annotation so GOMAXPROCS logic can call the narrow helper.

In `@server/container_create.go`:
- Around line 1044-1059: The current GOMAXPROCS injection decision uses CPU
resources read before NRI can mutate the spec (it reads linux.GetResources()
then calls calculateGOMAXPROCS and injectGOMAXPROCS), so change the logic to
evaluate quota/shares after the final resource mutators: either move the
skip/calculation/inject block to after s.nri.createContainer(...) or re-read the
final OCI resources from specgen (e.g., inspect specgen.Linux.Resources or call
linux.GetResources() again) immediately before calling calculateGOMAXPROCS and
injectGOMAXPROCS so the decision uses the final CPU quota/cpuset values (use
resources.GetCpuQuota()/GetCpuShares()).

---

Duplicate comments:
In `@internal/criocli/criocli.go`:
- Around line 1338-1341: The flag definition for cli.Int64Flag named
"inject-gomaxprocs" (using defConf.InjectGOMAXPROCS) has outdated usage text;
update the Usage string to state that this value is both the fallback for
best-effort pods and the minimum floor applied to burstable pods (and that 0
disables injection). Edit the Usage in the cli.Int64Flag for "inject-gomaxprocs"
to mention "minimum floor for burstable pods" alongside the best-effort fallback
and keep the "0 to disable" note.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 9f6135f2-272c-481b-96bb-4d685aba6476

📥 Commits

Reviewing files that changed from the base of the PR and between 4efa766 and 474093e.

📒 Files selected for processing (13)
  • completions/bash/crio
  • completions/fish/crio.fish
  • completions/zsh/_crio
  • docs/crio.8.md
  • docs/crio.conf.5.md
  • internal/criocli/criocli.go
  • pkg/annotations/v2/annotations.go
  • pkg/config/config.go
  • pkg/config/template.go
  • pkg/config/workloads.go
  • server/container_create.go
  • server/container_create_gomaxprocs_test.go
  • test/inject_gomaxprocs.bats
✅ Files skipped from review due to trivial changes (5)
  • completions/zsh/_crio
  • completions/fish/crio.fish
  • docs/crio.conf.5.md
  • completions/bash/crio
  • test/inject_gomaxprocs.bats
🚧 Files skipped from review as they are similar to previous changes (3)
  • pkg/annotations/v2/annotations.go
  • pkg/config/template.go
  • pkg/config/config.go

Comment thread pkg/config/workloads.go
Comment on lines +171 to +172
func (w Workloads) IsWorkloadPartitioned(sboxAnnotations map[string]string) bool {
return w.workloadGivenActivationAnnotation(sboxAnnotations) != nil

@coderabbitai coderabbitai Bot Mar 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Don't treat every activated workload as partitioned.

workloadGivenActivationAnnotation(...) matches any activated workload, but WorkloadConfig here also represents non-partitioning overrides like cpuquota, cpulimit, and allowed experimental annotations. Using that as a synonym for “workload-partitioned” will suppress GOMAXPROCS injection for unrelated workloads too. Narrow this to the actual partitioning signal, or rename the helper if this broader behavior is intentional.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/config/workloads.go` around lines 171 - 172, The IsWorkloadPartitioned
method currently treats any activation matched by
workloadGivenActivationAnnotation as "partitioned", which incorrectly flags
non-partitioning overrides (e.g., cpuquota/cpulimit or experimental annotations
in WorkloadConfig) and suppresses GOMAXPROCS injection; update
IsWorkloadPartitioned to check the real partitioning signal instead of the broad
helper—either inspect the returned WorkloadConfig for an explicit partitioning
field/property (e.g., a Partition or IsPartition boolean) or check for the
specific partition annotation key used in your annotations map; if the broader
behavior was intended, rename workloadGivenActivationAnnotation to reflect that
it matches any activation (e.g., workloadGivenActivationOrOverride) and add a
new helper (e.g., workloadIndicatesPartitioning) that only detects the
partitioning annotation so GOMAXPROCS logic can call the narrow helper.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CRI-O Workloads feature is specifically designed for workload partitioning — the activation annotation is target.workload.openshift.io/management, which is exclusively used for cpuset-based workload partitioning in OpenShift. There is no real-world use case where a workload config is activated without partitioning. The naming and behavior are correct for how this feature is actually used.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harche, thanks for the clarification — that context makes sense. Since target.workload.openshift.io/management is the sole activation annotation used in practice and it is exclusively tied to cpuset-based workload partitioning in OpenShift, the current IsWorkloadPartitioned implementation is correct and my concern doesn't apply. I'll record this for future reviews.


✏️ Learnings added
Learnt from: harche
Repo: cri-o/cri-o PR: 9844
File: pkg/config/workloads.go:171-172
Timestamp: 2026-03-27T13:03:11.501Z
Learning: In the cri-o/cri-o repository, the CRI-O `Workloads` feature (`pkg/config/workloads.go`) is exclusively used for cpuset-based workload partitioning in OpenShift. The activation annotation is `target.workload.openshift.io/management`. There is no real-world use case where a WorkloadConfig is activated without workload partitioning, so treating any activation matched by `workloadGivenActivationAnnotation` as "partitioned" (as done in `IsWorkloadPartitioned`) is correct and intentional. Do not flag this as incorrect or suggest narrowing the check.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.


🧠 Learnings used
Learnt from: bitoku
Repo: cri-o/cri-o PR: 9667
File: server/container_create.go:1233-1236
Timestamp: 2025-12-17T13:38:34.646Z
Learning: In the cri-o/cri-o repository, protobuf-generated Get* methods for k8s.io/cri-api types are nil-safe: if the receiver is nil, GetX() returns the zero value instead of panicking. Do not add explicit nil checks before chaining calls on such getters. Apply this guidance to all Go code that uses these generated getters across the codebase.

Learnt from: bitoku
Repo: cri-o/cri-o PR: 9676
File: internal/lib/stats/cgroup_stats_unsupported.go:1-7
Timestamp: 2025-12-18T13:28:24.244Z
Learning: In the cri-o/cri-o repository, for platform-specific types guarded by Go build tags (for example //go:build !linux), implement empty structs for unsupported platforms to permit compilation and clearly indicate the feature is not available rather than mirroring the Linux struct with unpopulated fields. Apply this pattern to all relevant platform-specific files across the codebase (i.e., any file under build-taged sections that should compile on all targets but lacks full implementation for some platforms).

Comment on lines +1044 to +1059
} else if cgroupParent := sb.CgroupParent(); strings.Contains(cgroupParent, "burstable") || strings.Contains(cgroupParent, "besteffort") {
var shares, cpuQuota int64

if linux != nil {
if resources := linux.GetResources(); resources != nil {
shares = resources.GetCpuShares()
cpuQuota = resources.GetCpuQuota()
}
}

// Skip if the container has a CPU limit (quota > 0). Go 1.25+ auto-detects
// GOMAXPROCS from the cgroup quota, so injecting would override that.
if cpuQuota <= 0 {
maxProcs := calculateGOMAXPROCS(shares, fallbackMaxProcs)

injectGOMAXPROCS(specgen, envs, s.ContainerServer.Config().DefaultEnv, maxProcs)

@coderabbitai coderabbitai Bot Mar 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Make this decision from the final CPU resources.

This runs before s.nri.createContainer(...), but NRI can still rewrite quota/cpuset on specgen later in the same create flow. In that case the container keeps an injected GOMAXPROCS even though Go would have derived the right value from the final cgroup limits. Please evaluate the skip/calculate logic after the last resource mutator, or re-read the final OCI resources before injecting.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@server/container_create.go` around lines 1044 - 1059, The current GOMAXPROCS
injection decision uses CPU resources read before NRI can mutate the spec (it
reads linux.GetResources() then calls calculateGOMAXPROCS and injectGOMAXPROCS),
so change the logic to evaluate quota/shares after the final resource mutators:
either move the skip/calculation/inject block to after
s.nri.createContainer(...) or re-read the final OCI resources from specgen
(e.g., inspect specgen.Linux.Resources or call linux.GetResources() again)
immediately before calling calculateGOMAXPROCS and injectGOMAXPROCS so the
decision uses the final CPU quota/cpuset values (use
resources.GetCpuQuota()/GetCpuShares()).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a concern. The GOMAXPROCS code reads CPU shares/quota from the linux parameter (*types.LinuxContainerConfig — the CRI request from kubelet), not from specgen (the OCI spec):

if linux != nil {
if resources := linux.GetResources(); resources != nil {
shares = resources.GetCpuShares()
cpuQuota = resources.GetCpuQuota()
}

NRI's createContainer operates exclusively on the OCI specgen:

cri-o/server/nri-api.go

Lines 105 to 114 in 474093e

func (a *nriAPI) createContainer(ctx context.Context, specgen *generate.Generator, criPod *sandbox.Sandbox, criCtr *oci.Container) error {
if !a.isEnabled() {
return nil
}
pod := nriPodSandbox(ctx, criPod)
ctr := &criContainer{
api: a,
ctr: criCtr,
spec: specgen.Config,

Since NRI mutates specgen and the GOMAXPROCS calculation reads from the CRI linux config, the ordering is irrelevant — the input data is stable regardless of when NRI runs.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@harche, you're absolutely right — thank you for the clear explanation. The GOMAXPROCS decision reads shares/cpuQuota from the linux *types.LinuxContainerConfig parameter (the kubelet CRI request), which is immutable and cannot be touched by NRI. NRI only mutates specgen (the OCI spec). The two data sources are independent, so the ordering relative to s.nri.createContainer(...) is irrelevant. My concern was mistaken.


✏️ Learnings added
Learnt from: harche
Repo: cri-o/cri-o PR: 9844
File: server/container_create.go:1044-1059
Timestamp: 2026-03-27T13:05:07.245Z
Learning: In cri-o/cri-o server/container_create.go, the GOMAXPROCS injection logic in `setupContainerEnvironmentAndWorkdir` reads CPU shares/quota from the `linux *types.LinuxContainerConfig` parameter (the immutable CRI request from kubelet), NOT from `specgen`. NRI's `createContainer` only mutates `specgen` (the OCI spec), so NRI cannot affect the CPU resource values used for the GOMAXPROCS calculation. Do not raise concerns about NRI ordering invalidating the GOMAXPROCS decision.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.


🧠 Learnings used
Learnt from: bitoku
Repo: cri-o/cri-o PR: 9667
File: server/container_create.go:1233-1236
Timestamp: 2025-12-17T13:38:34.646Z
Learning: In the cri-o/cri-o repository, protobuf-generated Get* methods for k8s.io/cri-api types are nil-safe: if the receiver is nil, GetX() returns the zero value instead of panicking. Do not add explicit nil checks before chaining calls on such getters. Apply this guidance to all Go code that uses these generated getters across the codebase.

Learnt from: bitoku
Repo: cri-o/cri-o PR: 9676
File: internal/lib/stats/cgroup_stats_unsupported.go:1-7
Timestamp: 2025-12-18T13:28:24.244Z
Learning: In the cri-o/cri-o repository, for platform-specific types guarded by Go build tags (for example //go:build !linux), implement empty structs for unsupported platforms to permit compilation and clearly indicate the feature is not available rather than mirroring the Linux struct with unpopulated fields. Apply this pattern to all relevant platform-specific files across the codebase (i.e., any file under build-taged sections that should compile on all targets but lacks full implementation for some platforms).

@harche harche force-pushed the inject-gomaxprocs-dynamic branch from 474093e to d6f20f6 Compare March 27, 2026 13:19
@openshift-ci openshift-ci Bot added dco-signoff: no Indicates the PR's author has not DCO signed all their commits. and removed dco-signoff: yes Indicates the PR's author has DCO signed all their commits. labels Mar 27, 2026
@harche harche force-pushed the inject-gomaxprocs-dynamic branch from d6f20f6 to b1109f2 Compare March 27, 2026 13:20
@openshift-ci openshift-ci Bot added dco-signoff: yes Indicates the PR's author has DCO signed all their commits. and removed dco-signoff: no Indicates the PR's author has not DCO signed all their commits. labels Mar 27, 2026
Signed-off-by: Harshal Patil <12152047+harche@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@harche harche force-pushed the inject-gomaxprocs-dynamic branch from b1109f2 to 49a75db Compare March 27, 2026 15:06
@openshift-ci

openshift-ci Bot commented Mar 27, 2026

Copy link
Copy Markdown
Contributor

@harche: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/ci-fedora-kata 49a75db link true /test ci-fedora-kata

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@harche

harche commented Mar 27, 2026

Copy link
Copy Markdown
Contributor Author

failed runs are not due to the changes in this PR.

@harche

harche commented Mar 28, 2026

Copy link
Copy Markdown
Contributor Author

/hold until we test this using machine config to ensure end to end flow is covered.

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 28, 2026
@harche

harche commented Mar 31, 2026

Copy link
Copy Markdown
Contributor Author

Closing in favor of #9860

@harche harche closed this Mar 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has DCO signed all their commits. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. release-note Denotes a PR that will be considered when it comes time to generate release notes.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants