feat(kubernetes): update supported versions to v1.30-v1.35#2073
Conversation
Update Kamaji from edge-25.4.1 to edge-26.2.4 which adds support for Kubernetes 1.35 (KubeadmVersion bumped from v1.33.0 to v1.35.0). - Update Dockerfile VERSION to edge-26.2.4 - Update vendored Helm charts from upstream - Remove 992.diff patch (label preservation fix merged upstream) - Regenerate disable-datastore-check.diff for new version Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
Update Kubernetes version matrix to match Talos 1.12 support range: - Add v1.35.0 (latest) and v1.34.4 - Update existing patch versions (v1.33.8, v1.32.12) - Drop EOL versions v1.28 and v1.29 - Set default version to v1.35 Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughBumps Kubernetes default to v1.35 and updates version maps; adds a new kamaji-crds Helm chart (DataStore and KubeconfigGenerator CRDs, templates, values, docs, webhook conversion); extends Kamaji chart with a kubeconfigGenerator deployment, RBAC updates, image/patch changes, konnectivity overrides, and various schema/value edits. Changes
Sequence DiagramsequenceDiagram
participant Admin as Administrator
participant KubeAPI as Kubernetes API
participant KG as KubeconfigGenerator Controller
participant TCP as TenantControlPlane
Admin->>KubeAPI: Create KubeconfigGenerator CR
KubeAPI->>KG: Watch/event triggers controller
KG->>KubeAPI: List Namespaces & TenantControlPlanes (selectors)
KubeAPI-->>KG: Return matches
KG->>TCP: Read TenantControlPlane metadata
TCP-->>KG: Return definitions
KG->>KG: Generate kubeconfigs for matched resources
KG->>KubeAPI: Update KubeconfigGenerator status (availableResources/resources)
KubeAPI-->>Admin: Status updated
Estimated Code Review Effort🎯 4 (Complex) | ⏱️ ~55 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Summary of ChangesHello Aleksei Sviridkin (@lexfrei), I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request focuses on modernizing the Kubernetes ecosystem by updating the supported Kubernetes versions to the latest stable releases (v1.30-v1.35) and removing older, unsupported versions. A significant upgrade to the Kamaji control plane manager is also included, which brings new Custom Resource Definitions and introduces the Kubeconfig Generator feature, enhancing the management capabilities for tenant control planes. These changes ensure compatibility with current Kubernetes landscapes and improve overall system functionality. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request updates the supported Kubernetes versions to v1.30-v1.35, dropping v1.28 and v1.29. This involves updating the default Kubernetes version, version matrices, and dependencies like Kamaji. The changes are consistent across Makefiles, documentation, and configuration files. A new kamaji-crds Helm chart is also introduced. I've found one area for improvement regarding maintainability.
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
packages/system/kamaji/images/kamaji/Dockerfile (1)
10-10:⚠️ Potential issue | 🟠 MajorMissing integrity check on the downloaded tarball — supply chain risk.
The
curl | tarpipeline extracts the source archive without any checksum or signature verification. Git tags on GitHub can be force-pushed to a different commit, and there is no detection mechanism in place: a silently tampered or replaced archive would be compiled and shipped without notice.Add a SHA-256 verification step after the download, pinning the expected digest alongside the version:
🛡️ Proposed fix to add checksum verification
-RUN curl -sSL https://github.com/clastix/kamaji/archive/refs/tags/${VERSION}.tar.gz | tar -xzvf- --strip=1 +ARG VERSION_SHA256="" +RUN curl -sSL https://github.com/clastix/kamaji/archive/refs/tags/${VERSION}.tar.gz \ + -o /tmp/kamaji.tar.gz \ + && echo "${VERSION_SHA256} /tmp/kamaji.tar.gz" | sha256sum -c - \ + && tar -xzvf /tmp/kamaji.tar.gz --strip=1 \ + && rm /tmp/kamaji.tar.gz🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/system/kamaji/images/kamaji/Dockerfile` at line 10, The Dockerfile currently pipes curl to tar without verifying the archive; modify the RUN that uses VERSION to instead download the ${VERSION}.tar.gz to a temporary file, compute its SHA-256 and compare it against a pinned expected digest (add a CHECKSUM or SHA256_<VERSION> variable alongside VERSION), fail build if the sums differ, and only then extract (tar -xzvf) the verified file and --strip=1; ensure the checksum compare is done with a safe comparison (e.g., sha256sum or shasum -a 256) so tampered archives abort the build.packages/apps/kubernetes/README.md (1)
15-15:⚠️ Potential issue | 🟡 MinorStale version range in introduction text.
Line 15 still reads
"from 1.28 to 1.33"but the support matrix now coversv1.30tov1.35. This introductory text is not regenerated bycozyvalues-gen(which only updates the Parameters tables), so it needs to be updated manually or in the README template.📝 Proposed fix
-Users can select the latest patch versions from 1.28 to 1.33. +Users can select the latest patch versions from 1.30 to 1.35.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/apps/kubernetes/README.md` at line 15, Update the README introduction text that currently reads "from 1.28 to 1.33" to reflect the current supported Kubernetes range "v1.30 to v1.35"; locate the string in packages/apps/kubernetes/README.md (the introductory sentence) and replace the old range with the new range, ensuring the wording/format (including the "v" prefix) matches the support matrix.
🧹 Nitpick comments (8)
packages/system/kamaji/images/kamaji/Dockerfile (1)
2-2: Floating base image tags reduce build reproducibility.Both
golang:1.24(line 2) andgcr.io/distroless/static:nonroot(line 21) are floating tags. Silent patch-level updates to either can change the compiled binary or the shipped runtime layer between two otherwise identical builds.Pinning both to immutable digests (
@sha256:...) is the standard mitigation for reproducible, auditable container builds:♻️ Proposed fix to pin base images by digest
-FROM golang:1.24 as builder +FROM golang:1.24@sha256:<digest> AS builder ... -FROM gcr.io/distroless/static:nonroot +FROM gcr.io/distroless/static:nonroot@sha256:<digest>Retrieve the current digests with:
docker buildx imagetools inspect golang:1.24 --format '{{json .Manifest.Digest}}' docker buildx imagetools inspect gcr.io/distroless/static:nonroot --format '{{json .Manifest.Digest}}'Also applies to: 21-21
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/system/kamaji/images/kamaji/Dockerfile` at line 2, The Dockerfile uses floating base tags which break reproducibility; pin both base images by replacing the identifiers "golang:1.24" (the builder stage FROM line) and "gcr.io/distroless/static:nonroot" (the final stage FROM) with their immutable digest forms (e.g. golang@sha256:... and gcr.io/distroless/static@sha256:...)—obtain the correct digests via `docker buildx imagetools inspect` as suggested and update those two FROM lines to the corresponding `@sha256` digests so builds are deterministic and auditable.packages/system/kamaji/charts/kamaji-crds/templates/kamaji.clastix.io_tenantcontrolplanes.yaml (1)
10-10: Use{{-to eliminate the spurious blank line underspec:.The two leading spaces before
{{are preserved in the output, andnindent 2prepends a newline, resulting in an empty line betweenspec:and the first CRD spec key. While valid YAML, using{{-strips the preceding whitespace sonindent 2alone provides the correct formatting.This pattern appears in all three CRD templates in this chart and should be updated consistently.
♻️ Proposed fix
spec: - {{ tpl (.Files.Get "hack/kamaji.clastix.io_tenantcontrolplanes_spec.yaml") . | nindent 2 }} +{{- tpl (.Files.Get "hack/kamaji.clastix.io_tenantcontrolplanes_spec.yaml") . | nindent 2 }}Apply the same change to
kamaji.clastix.io_datastores.yamlandkamaji.clastix.io_kubeconfiggenerators.yaml.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/system/kamaji/charts/kamaji-crds/templates/kamaji.clastix.io_tenantcontrolplanes.yaml` at line 10, Replace the current tpl invocation that leaves a spurious blank line after "spec:" by switching the opening action to a whitespace-trimming form; specifically update the template expression using tpl (.Files.Get "hack/kamaji.clastix.io_tenantcontrolplanes_spec.yaml") . | nindent 2 to use the left-trimmed action (change the leading "{{" to "{{-") so nindent 2 no longer inserts an extra blank line. Apply the same change to the equivalent tpl invocations in the other two CRD templates (kamaji.clastix.io_datastores.yaml and kamaji.clastix.io_kubeconfiggenerators.yaml) so all three use the left-trimmed "{{-" form.packages/apps/kubernetes/files/versions.yaml (1)
1-6: EOL status of included versions warrants an operator notice.Kubernetes 1.32 entered maintenance mode on Dec 28, 2025, with an End of Life date of Feb 28, 2026 — this month. Releases 1.31 and older are listed as no longer supported by the upstream project.
The Kubernetes project currently maintains release branches only for the most recent three minor releases (1.35, 1.34, 1.33).
v1.32 will stop receiving upstream security patches in ~9 days; v1.31 and v1.30 are already past upstream EOL. If Cozystack's support policy intentionally extends beyond upstream EOL (e.g., for migration runway), consider adding a deprecation notice in the UI/docs for these versions so operators know to plan upgrades.
Additionally, Kubernetes 1.35 is the last release supporting containerd 1.x. Users must switch to containerd 2.0 or later before upgrading to the next Kubernetes version. This is worth surfacing as a release note for anyone upgrading worker nodes to v1.35.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/apps/kubernetes/files/versions.yaml` around lines 1 - 6, The versions.yaml list includes v1.32, v1.31 and v1.30 which are at or past upstream EOL and v1.35 is the last to support containerd 1.x; update the repository so operators see this: add deprecation/EOL metadata or a linked notice for the keys "v1.32", "v1.31", and "v1.30" (e.g., a deprecation flag and EOL date) and add a release note/notice tied to "v1.35" informing users that containerd 1.x is supported only up to v1.35 and they must migrate to containerd 2.x+ before upgrading beyond v1.35; ensure this change is surfaced to UI/docs where versions.yaml is consumed.packages/system/kamaji/images/kamaji/patches/disable-datastore-check.diff (1)
1-24: Startup DataStore validation removed — confirm operational intent.The regenerated patch now removes the
datastoreutils.CheckExistscall entirely (previously it just switched context; now neither import nor call remains). This means Kamaji will no longer fail fast at startup if the configured DataStore is unavailable. The file name makes the intent clear, but ensure monitoring/alerting compensates for the removed fail-fast guard.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/system/kamaji/images/kamaji/patches/disable-datastore-check.diff` around lines 1 - 24, The startup DataStore existence check was removed (datastoreutils.CheckExists) causing Kamaji to no longer fail-fast if the configured DataStore is unavailable; restore the original behavior by re-adding the datastoreutils import and calling datastoreutils.CheckExists(ctx, scheme, datastore) inside NewCmd (before the controllerReconcileTimeout validation), returning the error if it fails; ensure the import alias matches datastoreutils and the error handling returns the error up the stack so startup fails fast when the DataStore is unreachable.packages/system/kamaji/charts/kamaji/Chart.yaml (1)
2-2:appVersion: latest— verify intentional upstream versioning strategy.Using
latestasappVersiondiverges from semver conventions (and from the previousv0.0.0). Given the ArtifactHub annotation at line 50 (Releasing latest chart at every push), this appears intentional for a rolling-latest track. Confirm this matches the intended upgrade path and that any downstream tooling (e.g., image-pinning logic) handles a non-semverappVersioncorrectly.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/system/kamaji/charts/kamaji/Chart.yaml` at line 2, The Chart.yaml currently sets appVersion: latest which breaks semver expectations; confirm whether the rolling-latest release strategy is intentional and either restore a semver value (e.g., vX.Y.Z) in appVersion or add clear documentation/annotations and update downstream tooling (image-pinning, upgrade checks, ArtifactHub integrations) to accept non-semver values; check and update any code paths that read Chart.yaml.appVersion (search for references to appVersion in Chart.yaml, the ArtifactHub annotation, image-pinning logic, and release scripts) to ensure they correctly handle the "latest" string or change it back to a proper semver.packages/system/kamaji/charts/kamaji/values.yaml (1)
115-158: Add configurablelivenessProbeandreadinessProbefields tokubeconfigGeneratorfor pattern consistency.The main Kamaji controller exposes
livenessProbeandreadinessProbeas user-configurable top-level values (lines 26–39), but thekubeconfigGeneratorblock lacks these fields. While the deployment template only setshealthProbeBindAddresswithout probe configuration, adding explicit probe values would align with the established pattern and allow users to customize probe behavior if needed.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/system/kamaji/charts/kamaji/values.yaml` around lines 115 - 158, Add configurable livenessProbe and readinessProbe fields under the kubeconfigGenerator values block so users can override probe behavior (similar to the top-level livenessProbe/readinessProbe). Specifically, in the kubeconfigGenerator section (which contains healthProbeBindAddress, replicaCount, serviceAccountOverride, etc.) add livenessProbe: {} and readinessProbe: {} with the same configurable shape used by the main controller, ensuring the deployment template consumes these new keys instead of only healthProbeBindAddress; update any references in the Kubeconfig Generator deployment manifest to use kubeconfigGenerator.livenessProbe and kubeconfigGenerator.readinessProbe.packages/system/kamaji/charts/kamaji/templates/kubeconfiggenerator-deployment.yaml (1)
27-41: Missing liveness and readiness probes despitehealthProbeBindAddressbeing configured.The container has
--health-probe-bind-addressconfigured but nolivenessProbeorreadinessProbeare defined, unlike the main Kamaji controller. Without probes, Kubernetes cannot detect an unhealthy container and will not restart or route traffic accordingly.♻️ Suggested addition (align with main Kamaji controller's probe config)
resources: {{- toYaml .Values.kubeconfigGenerator.resources | nindent 12 }} securityContext: {{- toYaml .Values.kubeconfigGenerator.securityContext | nindent 12 }} + livenessProbe: + httpGet: + path: /healthz + port: {{ .Values.kubeconfigGenerator.healthProbeBindAddress | trimPrefix ":" }} + initialDelaySeconds: 15 + periodSeconds: 20 + readinessProbe: + httpGet: + path: /readyz + port: {{ .Values.kubeconfigGenerator.healthProbeBindAddress | trimPrefix ":" }} + initialDelaySeconds: 5 + periodSeconds: 10🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/system/kamaji/charts/kamaji/templates/kubeconfiggenerator-deployment.yaml` around lines 27 - 41, Add livenessProbe and readinessProbe to the kubeconfig-generator container spec (the container named "controller") so Kubernetes can detect unhealthy instances; use HTTP probes that hit the health endpoints (e.g., /healthz for liveness and /readyz for readiness) on the same health port/address used by the --health-probe-bind-address flag (mirror the main Kamaji controller probe configuration: set initialDelaySeconds, periodSeconds, timeoutSeconds, failureThreshold/successThreshold), and place these probe blocks alongside the existing resources and securityContext in the container definition, wired to the chart values (e.g., .Values.kubeconfigGenerator.healthProbeBindAddress / .Values.kubeconfigGenerator.healthPort or default port) so they respect user-configured probe settings.packages/system/kamaji/charts/kamaji-crds/templates/_helpers.tpl (1)
13-24: Renamekamaji.fullnametokamaji-crds.fullnamefor naming consistency.This template defines
kamaji.fullnamebut it is never used in the chart. The definition breaks thekamaji-crds.*naming convention that all other helpers follow (kamaji-crds.name,kamaji-crds.chart,kamaji-crds.certManagerAnnotation,kamaji-crds.labels). Either rename it to match the convention or remove it if it's truly unused.♻️ Suggested rename
-{{- define "kamaji.fullname" -}} +{{- define "kamaji-crds.fullname" -}} {{- if .Values.fullnameOverride }} {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} {{- else }} {{- $name := default .Chart.Name .Values.nameOverride }} {{- if contains $name .Release.Name }} {{- .Release.Name | trunc 63 | trimSuffix "-" }} {{- else }} {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} {{- end }} {{- end }} {{- end }}🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/system/kamaji/charts/kamaji-crds/templates/_helpers.tpl` around lines 13 - 24, The helper currently defines "kamaji.fullname" which breaks the chart's "kamaji-crds.*" helper naming convention; rename the template definition from "kamaji.fullname" to "kamaji-crds.fullname" (or delete it if truly unused) and update any internal references to use "kamaji-crds.fullname" so it matches the existing helpers like "kamaji-crds.name", "kamaji-crds.chart", and "kamaji-crds.labels"; ensure the definition body (logic using .Values.fullnameOverride, $name, .Release.Name, trunc/trimSuffix) remains identical after the rename.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/apps/kubernetes/files/versions.yaml`:
- Line 1: The entry "v1.35": "v1.35.0" in versions.yaml is stale; re-run the
version update script (update-versions.sh) to regenerate versions.yaml so the
value becomes "v1.35.1". Run the script used to populate versions.yaml, commit
the regenerated file, and verify the updated key/value "v1.35": "v1.35.1" is
present.
In `@packages/system/kamaji/charts/kamaji-crds/Chart.yaml`:
- Around line 19-29: The artifacthub.io/crds annotation in Chart.yaml is missing
the KubeconfigGenerator CRD entry; update the annotation block that currently
lists TenantControlPlane and DataStore to also include a KubeconfigGenerator
entry (matching the other CRDs' format) so artifact hub can surface it—add a
list item with kind: KubeconfigGenerator, version: v1alpha1, name:
kubeconfiggenerators.kamaji.clastix.io, displayName: KubeconfigGenerator and a
brief description consistent with the existing entries.
In `@packages/system/kamaji/charts/kamaji-crds/Makefile`:
- Around line 1-9: Add a .PHONY declaration for the Makefile targets so make
always runs them: update the Makefile (targets "docs" and "docker") to include a
line like ".PHONY: docs docker" near the top so neither "docs" nor "docker" will
be treated as up-to-date files if a same-named artifact exists.
In `@packages/system/kamaji/charts/kamaji/templates/_helpers.tpl`:
- Around line 97-102: The template helper kamaji.kubeconfigGeneratorName
incorrectly uses .Values.fullnameOverride in the override branch; update the
override branch to use .Values.kubeconfigGenerator.fullnameOverride (the same
key checked in the if) so the generated name comes from
kubeconfigGenerator.fullnameOverride; ensure you still apply trunc 63 and
trimSuffix "-" to that value just like the else branch.
In
`@packages/system/kamaji/charts/kamaji/templates/kubeconfiggenerator-deployment.yaml`:
- Line 25: The serviceAccount override is never used because the Helm default
arguments are reversed; change the default call so the included name is the
first arg and the override (.Values.kubeconfigGenerator.serviceAccountOverride)
is the second so the override is returned when non-empty (i.e., replace default
.Values.kubeconfigGenerator.serviceAccountOverride (include
"kamaji.serviceAccountName" .) with default (include "kamaji.serviceAccountName"
.) .Values.kubeconfigGenerator.serviceAccountOverride to ensure
serviceAccountName honors the override).
- Around line 11-21: The selector collision is caused by both Deployments
calling the kamaji.selectorLabels helper with the same context, so update the
kubeconfigGenerator Deployment to call include "kamaji.selectorLabels" with an
overridden component (e.g., pass a dict with "component":"kubeconfig-generator")
for both the selector.matchLabels and the pod template labels; specifically
change the two occurrences of include "kamaji.selectorLabels" . to include
"kamaji.selectorLabels" (dict "component" "kubeconfig-generator" "Release" .) or
an equivalent dict-based context so the rendered app.kubernetes.io/component
becomes kubeconfig-generator and prevents ReplicaSet/pod collisions.
---
Outside diff comments:
In `@packages/apps/kubernetes/README.md`:
- Line 15: Update the README introduction text that currently reads "from 1.28
to 1.33" to reflect the current supported Kubernetes range "v1.30 to v1.35";
locate the string in packages/apps/kubernetes/README.md (the introductory
sentence) and replace the old range with the new range, ensuring the
wording/format (including the "v" prefix) matches the support matrix.
In `@packages/system/kamaji/images/kamaji/Dockerfile`:
- Line 10: The Dockerfile currently pipes curl to tar without verifying the
archive; modify the RUN that uses VERSION to instead download the
${VERSION}.tar.gz to a temporary file, compute its SHA-256 and compare it
against a pinned expected digest (add a CHECKSUM or SHA256_<VERSION> variable
alongside VERSION), fail build if the sums differ, and only then extract (tar
-xzvf) the verified file and --strip=1; ensure the checksum compare is done with
a safe comparison (e.g., sha256sum or shasum -a 256) so tampered archives abort
the build.
---
Nitpick comments:
In `@packages/apps/kubernetes/files/versions.yaml`:
- Around line 1-6: The versions.yaml list includes v1.32, v1.31 and v1.30 which
are at or past upstream EOL and v1.35 is the last to support containerd 1.x;
update the repository so operators see this: add deprecation/EOL metadata or a
linked notice for the keys "v1.32", "v1.31", and "v1.30" (e.g., a deprecation
flag and EOL date) and add a release note/notice tied to "v1.35" informing users
that containerd 1.x is supported only up to v1.35 and they must migrate to
containerd 2.x+ before upgrading beyond v1.35; ensure this change is surfaced to
UI/docs where versions.yaml is consumed.
In `@packages/system/kamaji/charts/kamaji-crds/templates/_helpers.tpl`:
- Around line 13-24: The helper currently defines "kamaji.fullname" which breaks
the chart's "kamaji-crds.*" helper naming convention; rename the template
definition from "kamaji.fullname" to "kamaji-crds.fullname" (or delete it if
truly unused) and update any internal references to use "kamaji-crds.fullname"
so it matches the existing helpers like "kamaji-crds.name", "kamaji-crds.chart",
and "kamaji-crds.labels"; ensure the definition body (logic using
.Values.fullnameOverride, $name, .Release.Name, trunc/trimSuffix) remains
identical after the rename.
In
`@packages/system/kamaji/charts/kamaji-crds/templates/kamaji.clastix.io_tenantcontrolplanes.yaml`:
- Line 10: Replace the current tpl invocation that leaves a spurious blank line
after "spec:" by switching the opening action to a whitespace-trimming form;
specifically update the template expression using tpl (.Files.Get
"hack/kamaji.clastix.io_tenantcontrolplanes_spec.yaml") . | nindent 2 to use the
left-trimmed action (change the leading "{{" to "{{-") so nindent 2 no longer
inserts an extra blank line. Apply the same change to the equivalent tpl
invocations in the other two CRD templates (kamaji.clastix.io_datastores.yaml
and kamaji.clastix.io_kubeconfiggenerators.yaml) so all three use the
left-trimmed "{{-" form.
In `@packages/system/kamaji/charts/kamaji/Chart.yaml`:
- Line 2: The Chart.yaml currently sets appVersion: latest which breaks semver
expectations; confirm whether the rolling-latest release strategy is intentional
and either restore a semver value (e.g., vX.Y.Z) in appVersion or add clear
documentation/annotations and update downstream tooling (image-pinning, upgrade
checks, ArtifactHub integrations) to accept non-semver values; check and update
any code paths that read Chart.yaml.appVersion (search for references to
appVersion in Chart.yaml, the ArtifactHub annotation, image-pinning logic, and
release scripts) to ensure they correctly handle the "latest" string or change
it back to a proper semver.
In
`@packages/system/kamaji/charts/kamaji/templates/kubeconfiggenerator-deployment.yaml`:
- Around line 27-41: Add livenessProbe and readinessProbe to the
kubeconfig-generator container spec (the container named "controller") so
Kubernetes can detect unhealthy instances; use HTTP probes that hit the health
endpoints (e.g., /healthz for liveness and /readyz for readiness) on the same
health port/address used by the --health-probe-bind-address flag (mirror the
main Kamaji controller probe configuration: set initialDelaySeconds,
periodSeconds, timeoutSeconds, failureThreshold/successThreshold), and place
these probe blocks alongside the existing resources and securityContext in the
container definition, wired to the chart values (e.g.,
.Values.kubeconfigGenerator.healthProbeBindAddress /
.Values.kubeconfigGenerator.healthPort or default port) so they respect
user-configured probe settings.
In `@packages/system/kamaji/charts/kamaji/values.yaml`:
- Around line 115-158: Add configurable livenessProbe and readinessProbe fields
under the kubeconfigGenerator values block so users can override probe behavior
(similar to the top-level livenessProbe/readinessProbe). Specifically, in the
kubeconfigGenerator section (which contains healthProbeBindAddress,
replicaCount, serviceAccountOverride, etc.) add livenessProbe: {} and
readinessProbe: {} with the same configurable shape used by the main controller,
ensuring the deployment template consumes these new keys instead of only
healthProbeBindAddress; update any references in the Kubeconfig Generator
deployment manifest to use kubeconfigGenerator.livenessProbe and
kubeconfigGenerator.readinessProbe.
In `@packages/system/kamaji/images/kamaji/Dockerfile`:
- Line 2: The Dockerfile uses floating base tags which break reproducibility;
pin both base images by replacing the identifiers "golang:1.24" (the builder
stage FROM line) and "gcr.io/distroless/static:nonroot" (the final stage FROM)
with their immutable digest forms (e.g. golang@sha256:... and
gcr.io/distroless/static@sha256:...)—obtain the correct digests via `docker
buildx imagetools inspect` as suggested and update those two FROM lines to the
corresponding `@sha256` digests so builds are deterministic and auditable.
In `@packages/system/kamaji/images/kamaji/patches/disable-datastore-check.diff`:
- Around line 1-24: The startup DataStore existence check was removed
(datastoreutils.CheckExists) causing Kamaji to no longer fail-fast if the
configured DataStore is unavailable; restore the original behavior by re-adding
the datastoreutils import and calling datastoreutils.CheckExists(ctx, scheme,
datastore) inside NewCmd (before the controllerReconcileTimeout validation),
returning the error if it fails; ensure the import alias matches datastoreutils
and the error handling returns the error up the stack so startup fails fast when
the DataStore is unreachable.
05234fc to
7115d10
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/system/kamaji/images/kamaji/Dockerfile (1)
2-2: Consider pinning to a specific Go patch version for reproducible builds.
golang:1.26is a floating tag. Once1.26.1is published, the tag resolves to a different layer, silently changing the build toolchain. Pinning togolang:1.26.0(or a digest) prevents inadvertent toolchain drift between builds.♻️ Proposed fix
-FROM golang:1.26 AS builder +FROM golang:1.26.0 AS builder🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/system/kamaji/images/kamaji/Dockerfile` at line 2, The Dockerfile's FROM instruction uses a floating Go tag ("FROM golang:1.26 AS builder"); change it to a pinned patch version (e.g., "golang:1.26.0") or a digest to ensure reproducible builds by preventing toolchain drift—update the FROM line in the Dockerfile accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@packages/system/kamaji/images/kamaji/Dockerfile`:
- Line 2: The Dockerfile's FROM instruction uses a floating Go tag ("FROM
golang:1.26 AS builder"); change it to a pinned patch version (e.g.,
"golang:1.26.0") or a digest to ensure reproducible builds by preventing
toolchain drift—update the FROM line in the Dockerfile accordingly.
7115d10 to
a67098d
Compare
…mport Kamaji edge-26.2.4 requires Go >= 1.25.0, update base image accordingly. Also remove unused "context" import from disable-datastore-check patch, since removing the CheckExists call was the only usage of that package. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
a67098d to
8c6c69c
Compare
CAPI Kamaji provider v0.15.0 is incompatible with Kamaji edge-26.2.4 due to the new dataStoreUsername field with XValidation rule. The provider's CreateOrUpdate drops the field (not in its Go types), triggering "unsetting the dataStoreUsername is not supported" error. This results in KamajiControlPlane staying INITIALIZED=false even though the underlying TenantControlPlane reaches Ready. v0.16.0 includes support for DataStoreUsername (PR #243 in v0.15.4) and updated Kamaji types compatible with edge-26.2.4. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
packages/system/kamaji/images/kamaji/Dockerfile (1)
2-2: Consider pinning to a specific patch-version tag for reproducible builds
golang:1.25is a floating minor-version tag that will silently resolve to different patch images over time, making builds non-reproducible. Pinning to a specific patch (e.g.,golang:1.25.7) or an image digest ensures the same toolchain is used on every build.♻️ Proposed fix (pin to specific patch version)
-FROM golang:1.25 AS builder +FROM golang:1.25.7 AS builderOr, for maximum reproducibility, use a digest:
-FROM golang:1.25 AS builder +FROM golang:1.25.7@sha256:<digest> AS builder🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/system/kamaji/images/kamaji/Dockerfile` at line 2, Update the Docker base image specification to pin the floating tag "FROM golang:1.25" to a specific patch version or digest so builds are reproducible; replace the floating tag with a concrete patched tag like "golang:1.25.7" or use an image digest (sha256) in the same FROM instruction to ensure the same toolchain is used for every build.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/system/kamaji/images/kamaji/Dockerfile`:
- Line 2: Replace the floating Go base image tag in the Dockerfile (the FROM
instruction using golang:1.25) with a specific patch-pinned tag (for example
golang:1.25.7) to ensure reproducible builds; update the FROM line in
packages/system/kamaji/images/kamaji/Dockerfile and confirm the exact patch
version with the PR author to match Kamaji edge-26.2.4 requirements.
---
Nitpick comments:
In `@packages/system/kamaji/images/kamaji/Dockerfile`:
- Line 2: Update the Docker base image specification to pin the floating tag
"FROM golang:1.25" to a specific patch version or digest so builds are
reproducible; replace the floating tag with a concrete patched tag like
"golang:1.25.7" or use an image digest (sha256) in the same FROM instruction to
ensure the same toolchain is used for every build.
| @@ -1,7 +1,7 @@ | |||
| # Build the manager binary | |||
| FROM golang:1.24 as builder | |||
| FROM golang:1.25 AS builder | |||
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# Check the actual Dockerfile content
cat packages/system/kamaji/images/kamaji/Dockerfile | head -10Repository: cozystack/cozystack
Length of output: 298
🏁 Script executed:
# Find the git commit message for this PR to verify the stated intent
git log --oneline -20 | head -20Repository: cozystack/cozystack
Length of output: 130
🏁 Script executed:
# Also check if there's any documentation about why golang:1.25 was chosen
rg -i "golang.*1\.25|golang.*1\.26" packages/system/kamaji --type-listRepository: cozystack/cozystack
Length of output: 452
🏁 Script executed:
# Search for commits that modified the Kamaji Dockerfile
git log --oneline -- packages/system/kamaji/images/kamaji/Dockerfile | head -10Repository: cozystack/cozystack
Length of output: 130
🏁 Script executed:
# Search for any references to golang version changes or Kamaji version updates in recent commits
git log --oneline --all | grep -i "kamaji\|golang" | head -20Repository: cozystack/cozystack
Length of output: 1187
🏁 Script executed:
# Check if there are any comments or documentation about the golang version choice
rg "golang" packages/system/kamaji --type dockerfile -A 2 -B 2Repository: cozystack/cozystack
Length of output: 97
🏁 Script executed:
# Check the full git history of the Dockerfile to see previous golang versions
git log -p -- packages/system/kamaji/images/kamaji/Dockerfile | grep -A 5 -B 5 "golang:" | head -50Repository: cozystack/cozystack
Length of output: 45
🏁 Script executed:
# Search for any Kamaji version or golang version requirements in the codebase
rg "edge-26\.2\.4" packages/system/kamajiRepository: cozystack/cozystack
Length of output: 134
🏁 Script executed:
# Check if there are any build or requirements documentation
find packages/system/kamaji -name "*.md" -o -name "*.txt" -o -name "go.mod" | head -20Repository: cozystack/cozystack
Length of output: 270
🏁 Script executed:
# Check if Kamaji repository or any documentation mentions golang version requirements
rg "golang|Go.*version" packages/system/kamaji/charts/kamaji/README.mdRepository: cozystack/cozystack
Length of output: 45
🏁 Script executed:
# Search the Kamaji tarball source extraction for any golang version requirements
# First, let's look at what gets extracted from the Kamaji release
rg "go.mod|golang" packages/system/kamaji --type-list 2>/dev/null || find packages/system/kamaji -name "go.mod"Repository: cozystack/cozystack
Length of output: 45
🏁 Script executed:
# Verify golang docker image tags using a web search approach - check if golang:1.26 exists
curl -s "https://registry.hub.docker.com/v2/repositories/library/golang/tags/?page_size=100" | python3 << 'EOF'
import sys, json
try:
data = json.load(sys.stdin)
results = data.get('results', [])
golang_versions = [t['name'] for t in results if t['name'] in ['1.25', '1.26', '1.25.7', '1.26.0']]
print("Found golang Docker tags:", golang_versions)
except Exception as e:
print(f"Error: {e}")
EOFRepository: cozystack/cozystack
Length of output: 111
Pin golang base image to specific patch version for reproducibility
The Dockerfile uses golang:1.25, a floating minor-version tag that resolves to the latest 1.25.x patch at build time, preventing reproducible builds. Consider pinning to a specific patch version (e.g., golang:1.25.7) to ensure consistent builds across environments.
Additionally, verify the intended golang version with the PR author. The current value golang:1.25 satisfies Kamaji edge-26.2.4's requirements, but clarification on the golang version choice would be helpful.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/system/kamaji/images/kamaji/Dockerfile` at line 2, Replace the
floating Go base image tag in the Dockerfile (the FROM instruction using
golang:1.25) with a specific patch-pinned tag (for example golang:1.25.7) to
ensure reproducible builds; update the FROM line in
packages/system/kamaji/images/kamaji/Dockerfile and confirm the exact patch
version with the PR author to match Kamaji edge-26.2.4 requirements.
Go 1.26 was released on 2026-02-10 and is fully compatible with Kamaji edge-26.2.4 (which requires go 1.25.0 in go.mod). Verified by local build. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
v1.35.1 was released on 2026-02-10. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
Kamaji auto-derives the konnectivity proxy image tag as v0.{minor}.0
from the Kubernetes version. For K8s v1.35, this produces v0.35.0,
but the kas-network-proxy/proxy-server:v0.35.0 image does not exist
in registry.k8s.io yet, causing ImagePullBackOff on new TCP pods.
Add konnectivity-versions.yaml mapping to explicitly override the
konnectivity version when the auto-derived tag is unavailable.
For v1.35, pin to v0.34.0 (latest available).
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
When running kubernetes-latest and kubernetes-previous E2E tests simultaneously, worker VMs compete for resources in the sandbox environment. 3 minutes was insufficient for nodes to boot and join the tenant cluster under load. Increase to 5 minutes. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
hack/e2e-apps/run-kubernetes.sh (2)
74-75:⚠️ Potential issue | 🟡 MinorStale comment — says "2 minutes" but the actual timeout is
5m.🔧 Proposed fix
- # Wait for Kubernetes resources to be ready (timeout after 2 minutes) + # Wait for Kubernetes resources to be ready (timeout after 5 minutes) kubectl wait tcp -n tenant-test kubernetes-${test_name} --timeout=5m --for=jsonpath='{.status.kubernetesResources.version.status}'=Ready🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@hack/e2e-apps/run-kubernetes.sh` around lines 74 - 75, Update the stale inline comment above the kubectl wait invocation: change the comment string "# Wait for Kubernetes resources to be ready (timeout after 2 minutes)" to reflect the actual timeout used by the kubectl command (5 minutes). The relevant snippet to update is the comment immediately preceding the kubectl wait tcp -n tenant-test kubernetes-${test_name} --timeout=5m --for=jsonpath='{.status.kubernetesResources.version.status}'=Ready so the comment text and the --timeout value are consistent.
212-221:⚠️ Potential issue | 🟡 MinorPre-existing bug:
curlsucceeding on the 20th attempt is misreported as failure.After
break,$iis still20, so[ "$i" -eq 20 ]evaluates to true even on a successful last-attempt hit. A simple fix is to track success with a flag, or to use$?/ a sentinel instead.🐛 Proposed fix
+ lb_ok=false for i in $(seq 1 20); do echo "Attempt $i" - curl --silent --fail "http://${LB_ADDR}" && break + if curl --silent --fail "http://${LB_ADDR}"; then + lb_ok=true + break + fi sleep 3 done - if [ "$i" -eq 20 ]; then + if [ "$lb_ok" != "true" ]; then echo "LoadBalancer not reachable" >&2 exit 1 fi🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@hack/e2e-apps/run-kubernetes.sh` around lines 212 - 221, The loop that polls the load balancer uses the loop index i to determine failure, but i remains 20 even when curl succeeds on the last iteration, causing a false failure; update the script to track success with a sentinel (e.g., initialize LB_REACHABLE=0 before the for-loop, set LB_REACHABLE=1 when curl --silent --fail succeeds and then break) and after the loop test the sentinel (LB_REACHABLE) instead of comparing i to 20 so a success on the final attempt is reported correctly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@hack/e2e-apps/run-kubernetes.sh`:
- Around line 74-75: Update the stale inline comment above the kubectl wait
invocation: change the comment string "# Wait for Kubernetes resources to be
ready (timeout after 2 minutes)" to reflect the actual timeout used by the
kubectl command (5 minutes). The relevant snippet to update is the comment
immediately preceding the kubectl wait tcp -n tenant-test
kubernetes-${test_name} --timeout=5m
--for=jsonpath='{.status.kubernetesResources.version.status}'=Ready so the
comment text and the --timeout value are consistent.
- Around line 212-221: The loop that polls the load balancer uses the loop index
i to determine failure, but i remains 20 even when curl succeeds on the last
iteration, causing a false failure; update the script to track success with a
sentinel (e.g., initialize LB_REACHABLE=0 before the for-loop, set
LB_REACHABLE=1 when curl --silent --fail succeeds and then break) and after the
loop test the sentinel (LB_REACHABLE) instead of comparing i to 20 so a success
on the final attempt is reported correctly.
- Kill stale port-forward processes before starting a new one; on retries, the previous attempt's port-forward still holds the port, causing all kubectl commands to get "connection refused" - Use -ge 2 instead of -eq 2 for node count check; MachineHealthCheck may create a 3rd VM, leading to 3 nodes joining the tenant cluster which would never satisfy the exact equality check - Increase node join timeout from 5m to 8m; QEMU VMs with v1.34 need more time to boot and join when running after kubernetes-latest Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
Kamaji edge-26.2.4 is compiled against Kubernetes 1.35 libraries. SetDefaults_KubeletConfiguration() from 1.35 injects two fields gated by feature gates that are not enabled in earlier versions: - crashLoopBackOff.maxContainerRestartPeriod (KubeletCrashLoopBackOffMax) - imagePullCredentialsVerificationPolicy (KubeletEnsureSecretPulledImages) Kubelets < 1.35 reject these fields during configuration validation, causing worker nodes to fail to join the tenant cluster. Add a Go patch that clears these fields from the kubelet-config ConfigMap when the target Kubernetes version is below 1.35. See: clastix/kamaji#1062 Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Aleksei Sviridkin <f@lex.la>
72e4593 to
376e4d1
Compare
What this PR does
Updates Kubernetes version support to match current release landscape and Talos 1.12 compatibility.
Kubernetes versions
Kamaji
edge-25.4.1toedge-26.2.4(adds K8s 1.35 support)KubeletCrashLoopBackOffMaxandKubeletEnsureSecretPulledImagesfeature gates; kubelets < 1.35 reject these fields, preventing nodes from joining. Added Go patch to strip incompatible fields when target version < 1.35 (kamaji#1062)CAPI Kamaji provider
dataStoreUsernamefield with XValidation rule that prevents unsetting; the old provider v0.15.0 would drop this field on updates, causing validation failuresKonnectivity version pinning
v0.{minor}.0from K8s versionv0.35.0which does not exist inregistry.k8s.ioyetkonnectivity-versions.yamlmapping to pin v1.35 tov0.34.0(latest available)E2E test improvements
-ge 2instead of-eq 2for node count check to handle MachineHealthCheck creating extra VMsRelease note
Summary by CodeRabbit
New Features
Chores