Skip to content

feat(kubernetes): update supported versions to v1.30-v1.35#2073

Merged
Andrei Kvapil (kvaps) merged 10 commits into
mainfrom
feat/update-kubernetes-1.35
Feb 20, 2026
Merged

feat(kubernetes): update supported versions to v1.30-v1.35#2073
Andrei Kvapil (kvaps) merged 10 commits into
mainfrom
feat/update-kubernetes-1.35

Conversation

@lexfrei

@lexfrei Aleksei Sviridkin (lexfrei) commented Feb 19, 2026

Copy link
Copy Markdown
Contributor

What this PR does

Updates Kubernetes version support to match current release landscape and Talos 1.12 compatibility.

Kubernetes versions

  • Update version matrix: v1.30, v1.31, v1.32, v1.33, v1.34, v1.35
  • Default Kubernetes version is now v1.35
  • Drop EOL versions v1.28 and v1.29
  • Patch versions updated to latest available (v1.35.1, v1.34.4, v1.33.8, v1.32.12, v1.31.14, v1.30.14)

Kamaji

  • Update from edge-25.4.1 to edge-26.2.4 (adds K8s 1.35 support)
  • Update Go builder image to 1.26
  • Remove merged-upstream patch (992.diff — label preservation fix)
  • Regenerate disable-datastore-check.diff for new Kamaji version
  • Fix kubelet-config compatibility: Kamaji compiled with K8s 1.35 libraries injects fields gated behind KubeletCrashLoopBackOffMax and KubeletEnsureSecretPulledImages feature gates; kubelets < 1.35 reject these fields, preventing nodes from joining. Added Go patch to strip incompatible fields when target version < 1.35 (kamaji#1062)

CAPI Kamaji provider

  • Update from v0.15.0 to v0.16.0 (compatibility with Kamaji edge-26.2.4)
  • Required because Kamaji edge-26.2.4 introduced dataStoreUsername field with XValidation rule that prevents unsetting; the old provider v0.15.0 would drop this field on updates, causing validation failures

Konnectivity version pinning

  • Kamaji auto-derives konnectivity proxy image tag as v0.{minor}.0 from K8s version
  • For K8s v1.35, this produces v0.35.0 which does not exist in registry.k8s.io yet
  • Added konnectivity-versions.yaml mapping to pin v1.35 to v0.34.0 (latest available)
  • Override applies to both server and agent components

E2E test improvements

  • Kill stale port-forward processes before retries to prevent "connection refused" errors
  • Use -ge 2 instead of -eq 2 for node count check to handle MachineHealthCheck creating extra VMs
  • Increase worker node join timeout from 3m to 8m for QEMU environments with resource pressure

Release note

[kubernetes] Update supported Kubernetes versions to v1.30-v1.35

Summary by CodeRabbit

  • New Features

    • Kubernetes v1.35 and v1.34 now supported across configurations
    • Added KubeconfigGenerator custom resource for kubeconfig management
    • Konnectivity proxy versions now configurable per Kubernetes release
    • New kamaji-crds Helm chart with DataStore and KubeconfigGenerator CRDs
  • Chores

    • Updated default Kubernetes version to v1.35
    • Bumped provider version from v0.15.1 to v0.16.0
    • Updated Kamaji chart dependencies and removed EOL versions

Update Kamaji from edge-25.4.1 to edge-26.2.4 which adds support for
Kubernetes 1.35 (KubeadmVersion bumped from v1.33.0 to v1.35.0).

- Update Dockerfile VERSION to edge-26.2.4
- Update vendored Helm charts from upstream
- Remove 992.diff patch (label preservation fix merged upstream)
- Regenerate disable-datastore-check.diff for new version

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Update Kubernetes version matrix to match Talos 1.12 support range:
- Add v1.35.0 (latest) and v1.34.4
- Update existing patch versions (v1.33.8, v1.32.12)
- Drop EOL versions v1.28 and v1.29
- Set default version to v1.35

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
@coderabbitai

coderabbitai Bot commented Feb 19, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Bumps Kubernetes default to v1.35 and updates version maps; adds a new kamaji-crds Helm chart (DataStore and KubeconfigGenerator CRDs, templates, values, docs, webhook conversion); extends Kamaji chart with a kubeconfigGenerator deployment, RBAC updates, image/patch changes, konnectivity overrides, and various schema/value edits.

Changes

Cohort / File(s) Summary
Kubernetes version files
packages/apps/kubernetes/Makefile, packages/apps/kubernetes/README.md, packages/apps/kubernetes/files/versions.yaml, packages/apps/kubernetes/values.schema.json, packages/apps/kubernetes/values.yaml
Default Kubernetes version bumped from v1.33 to v1.35; added v1.34/v1.35 mappings and enums; removed older entries (v1.28, v1.29).
Konnectivity override & templates
packages/apps/kubernetes/files/konnectivity-versions.yaml, packages/apps/kubernetes/templates/_versions.tpl, packages/apps/kubernetes/templates/cluster.yaml
Added konnectivity per-version mapping and a template helper to apply the override conditionally to addons.konnectivity.
New kamaji-crds chart (full)
packages/system/kamaji/charts/kamaji-crds/Chart.yaml, .../README.md, .../README.md.gotmpl, .../.helmignore, .../Makefile, .../NOTES.txt, .../values.yaml, .../templates/*, .../hack/*, .../templates/_helpers.tpl
Introduces a dedicated Helm chart for Kamaji CRDs: chart metadata, docs/templates/helpers, values, helmignore, Makefile, NOTES, webhook conversion config, and CRD spec fragments.
CRD manifests & spec fragments
packages/system/kamaji/charts/kamaji/crds/kamaji.clastix.io_datastores.yaml, packages/system/kamaji/charts/kamaji/crds/kamaji.clastix.io_kubeconfiggenerators.yaml, packages/system/kamaji/charts/kamaji-crds/hack/*.yaml
Added KubeconfigGenerator CRD (full OpenAPI v3 schema, status, validations) and DataStore CRD spec fragments; DataStore CRD also gained observedGeneration and controller-gen annotation bump.
Kamaji chart core, templates & values
packages/system/kamaji/charts/kamaji/Chart.yaml, packages/system/kamaji/charts/kamaji/README.md, packages/system/kamaji/charts/kamaji/values.yaml, packages/system/kamaji/charts/kamaji/.helmignore, packages/system/kamaji/charts/kamaji/templates/_helpers.tpl, packages/system/kamaji/charts/kamaji/templates/kubeconfiggenerator-deployment.yaml
Updated chart metadata and dependency (kamaji-etcd >=0.11.0); added kubeconfigGenerator values and a Deployment template plus helper kamaji.kubeconfigGeneratorName; README/values updated to surface new subchart configuration.
RBAC / controller roles
packages/system/kamaji/charts/kamaji/controller-gen/clusterrole.yaml
Expanded ClusterRole to include gateway.networking.k8s.io resources and permissions for kubeconfiggenerators; adjusted resource group mappings and finalizers.
Kamaji image & code patches
packages/system/kamaji/images/kamaji/Dockerfile, packages/system/kamaji/images/kamaji/patches/992.diff, packages/system/kamaji/images/kamaji/patches/disable-datastore-check.diff
Updated Go builder base image and ARG; label mutation logic changed to merge existing labels with Kamaji labels (preserve existing labels); adjusted datastore existence check context/imports.
Misc schema/values & tooling
packages/system/kubernetes-rd/cozyrds/kubernetes.yaml, packages/system/capi-providers-cpprovider/files/metadata.yaml, .../templates/configmaps.yaml, .../templates/providers.yaml, hack/e2e-apps/run-kubernetes.sh
Chart values schema updated to include v1.35 and remove older entries; added capi provider releaseSeries 0.16 and bumped provider/configmap version strings; increased e2e node-ready timeout to 5 minutes.
New helper/templates & docs removal
packages/system/kamaji/charts/kamaji/app-readme.md
Removed legacy app-readme content.

Sequence Diagram

sequenceDiagram
    participant Admin as Administrator
    participant KubeAPI as Kubernetes API
    participant KG as KubeconfigGenerator Controller
    participant TCP as TenantControlPlane

    Admin->>KubeAPI: Create KubeconfigGenerator CR
    KubeAPI->>KG: Watch/event triggers controller
    KG->>KubeAPI: List Namespaces & TenantControlPlanes (selectors)
    KubeAPI-->>KG: Return matches
    KG->>TCP: Read TenantControlPlane metadata
    TCP-->>KG: Return definitions
    KG->>KG: Generate kubeconfigs for matched resources
    KG->>KubeAPI: Update KubeconfigGenerator status (availableResources/resources)
    KubeAPI-->>Admin: Status updated
Loading

Estimated Code Review Effort

🎯 4 (Complex) | ⏱️ ~55 minutes

Poem

🐇 I hop through charts with cheerful cheer,

v1.35 now springs near,
CRDs planted, kubeconfigs bloom,
Labels merge to save the room,
Tiny paws tidy code's new room.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The PR title 'feat(kubernetes): update supported versions to v1.30-v1.35' accurately summarizes the primary change: updating Kubernetes version support across multiple files and components.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feat/update-kubernetes-1.35

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello Aleksei Sviridkin (@lexfrei), I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on modernizing the Kubernetes ecosystem by updating the supported Kubernetes versions to the latest stable releases (v1.30-v1.35) and removing older, unsupported versions. A significant upgrade to the Kamaji control plane manager is also included, which brings new Custom Resource Definitions and introduces the Kubeconfig Generator feature, enhancing the management capabilities for tenant control planes. These changes ensure compatibility with current Kubernetes landscapes and improve overall system functionality.

Highlights

  • Kubernetes Version Support: Updated supported Kubernetes versions to include v1.30 through v1.35, while deprecating and removing support for EOL versions v1.28 and v1.29. The default Kubernetes version is now set to v1.35.
  • Kamaji Upgrade and Kubeconfig Generator: Upgraded Kamaji from edge-25.4.1 to edge-26.2.4, which introduces new Custom Resource Definitions (CRDs) and a dedicated Helm chart for kamaji-crds. This upgrade enables the new Kubeconfig Generator functionality.
  • Helm Chart and Patch Management: Added new Helm chart definitions and associated files for kamaji-crds, updated the kamaji-etcd dependency, and regenerated the disable-datastore-check.diff patch. An upstream-merged patch (992.diff) was removed.
  • Role-Based Access Control (RBAC) Enhancements: Expanded the cluster role permissions for Kamaji to include new resources such as Gateway API and KubeconfigGenerator CRDs, ensuring proper operation with the updated Kamaji version.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • packages/apps/kubernetes/Makefile
    • Updated the default Kubernetes version to v1.35.
  • packages/apps/kubernetes/README.md
    • Updated the default Kubernetes version in the documentation to v1.35.
  • packages/apps/kubernetes/files/versions.yaml
    • Added Kubernetes versions v1.34 and v1.35 with their latest patch releases.
    • Removed EOL Kubernetes versions v1.28 and v1.29.
  • packages/apps/kubernetes/values.schema.json
    • Updated the default Kubernetes version to v1.35.
    • Adjusted the allowed enum values to reflect supported versions v1.30-v1.35.
  • packages/apps/kubernetes/values.yaml
    • Updated the default Kubernetes version to v1.35.
    • Modified the enum values to support v1.30-v1.35.
  • packages/system/kamaji/charts/kamaji-crds/.helmignore
    • Added a new Helm ignore file for the kamaji-crds chart.
  • packages/system/kamaji/charts/kamaji-crds/Chart.yaml
    • Added a new Helm chart definition for Kamaji Custom Resource Definitions.
  • packages/system/kamaji/charts/kamaji-crds/Makefile
    • Added a new Makefile for generating Helm documentation for the kamaji-crds chart.
  • packages/system/kamaji/charts/kamaji-crds/NOTES.txt
    • Added a new NOTES.txt file for the kamaji-crds Helm chart.
  • packages/system/kamaji/charts/kamaji-crds/README.md
    • Added a new README file for the kamaji-crds Helm chart.
  • packages/system/kamaji/charts/kamaji-crds/README.md.gotmpl
    • Added a new Helm README template for the kamaji-crds chart.
  • packages/system/kamaji/charts/kamaji-crds/hack/crd-conversion.yaml
    • Added a new CRD conversion webhook configuration.
  • packages/system/kamaji/charts/kamaji-crds/hack/kamaji.clastix.io_datastores_spec.yaml
    • Added a new DataStore Custom Resource Definition specification.
  • packages/system/kamaji/charts/kamaji-crds/hack/kamaji.clastix.io_kubeconfiggenerators_spec.yaml
    • Added a new KubeconfigGenerator Custom Resource Definition specification.
  • packages/system/kamaji/charts/kamaji-crds/templates/_helpers.tpl
    • Added new Helm helper templates for the kamaji-crds chart.
  • packages/system/kamaji/charts/kamaji-crds/templates/kamaji.clastix.io_datastores.yaml
    • Added a new template for the DataStore Custom Resource Definition.
  • packages/system/kamaji/charts/kamaji-crds/templates/kamaji.clastix.io_kubeconfiggenerators.yaml
    • Added a new template for the KubeconfigGenerator Custom Resource Definition.
  • packages/system/kamaji/charts/kamaji-crds/templates/kamaji.clastix.io_tenantcontrolplanes.yaml
    • Added a new template for the TenantControlPlane Custom Resource Definition.
  • packages/system/kamaji/charts/kamaji-crds/values.yaml
    • Added a new values file for the kamaji-crds Helm chart.
  • packages/system/kamaji/charts/kamaji/.helmignore
    • Updated the Helm ignore file to include new patterns.
  • packages/system/kamaji/charts/kamaji/Chart.lock
    • Updated the kamaji-etcd dependency version to 0.11.0.
  • packages/system/kamaji/charts/kamaji/Chart.yaml
    • Updated the appVersion to latest and the chart version to 0.0.0+latest.
    • Updated the kamaji-etcd dependency to >=0.11.0.
    • Added a new artifacthub.io change entry for releasing the latest chart at every push.
  • packages/system/kamaji/charts/kamaji/README.md
    • Updated version badges and the kamaji-etcd dependency version in the README.
    • Added new values table entries for kubeconfigGenerator.
  • packages/system/kamaji/charts/kamaji/app-readme.md
    • Removed the app-readme.md file.
  • packages/system/kamaji/charts/kamaji/controller-gen/clusterrole.yaml
    • Added new permissions for core Kubernetes resources (configmaps, secrets, services, namespaces).
    • Added permissions for Gateway API resources (gateways, grpcroutes, httproutes, tlsroutes).
    • Added permissions for Kamaji KubeconfigGenerator CRDs and their status/finalizers.
  • packages/system/kamaji/charts/kamaji/crds/kamaji.clastix.io_datastores.yaml
    • Updated the controller-gen version to v0.20.0.
    • Added observedGeneration to the DataStore status schema.
  • packages/system/kamaji/charts/kamaji/crds/kamaji.clastix.io_kubeconfiggenerators.yaml
    • Added a new Custom Resource Definition for KubeconfigGenerators.
  • packages/system/kamaji/charts/kamaji/templates/_helpers.tpl
    • Added a new helper template for kamaji.kubeconfigGeneratorName.
  • packages/system/kamaji/charts/kamaji/templates/kubeconfiggenerator-deployment.yaml
    • Added a new deployment template for the Kubeconfig Generator.
  • packages/system/kamaji/charts/kamaji/values.yaml
    • Added clusterDomain to kamaji-etcd configuration.
    • Introduced a new kubeconfigGenerator section with various configuration options.
  • packages/system/kamaji/images/kamaji/Dockerfile
    • Updated the Kamaji base image version to edge-26.2.4.
  • packages/system/kamaji/images/kamaji/patches/992.diff
    • Removed the 992.diff patch file.
  • packages/system/kamaji/images/kamaji/patches/disable-datastore-check.diff
    • Regenerated the patch file to disable datastore checks.
  • packages/system/kubernetes-rd/cozyrds/kubernetes.yaml
    • Updated the Kubernetes version enum and default in the OpenAPI schema to support v1.30-v1.35.
Activity
  • The pull request was opened by lexfrei.
  • No human review comments or approvals have been recorded yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment Gemini (@gemini-code-assist) Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on Gemini (@gemini-code-assist) comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the supported Kubernetes versions to v1.30-v1.35, dropping v1.28 and v1.29. This involves updating the default Kubernetes version, version matrices, and dependencies like Kamaji. The changes are consistent across Makefiles, documentation, and configuration files. A new kamaji-crds Helm chart is also introduced. I've found one area for improvement regarding maintainability.

Comment thread packages/system/kubernetes-rd/cozyrds/kubernetes.yaml
@lexfrei Aleksei Sviridkin (lexfrei) marked this pull request as ready for review February 19, 2026 13:06
@dosubot dosubot Bot added size/XXL This PR changes 1000+ lines, ignoring generated files kind/feature Categorizes issue or PR as related to a new feature labels Feb 19, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
packages/system/kamaji/images/kamaji/Dockerfile (1)

10-10: ⚠️ Potential issue | 🟠 Major

Missing integrity check on the downloaded tarball — supply chain risk.

The curl | tar pipeline extracts the source archive without any checksum or signature verification. Git tags on GitHub can be force-pushed to a different commit, and there is no detection mechanism in place: a silently tampered or replaced archive would be compiled and shipped without notice.

Add a SHA-256 verification step after the download, pinning the expected digest alongside the version:

🛡️ Proposed fix to add checksum verification
-RUN curl -sSL https://github.com/clastix/kamaji/archive/refs/tags/${VERSION}.tar.gz | tar -xzvf- --strip=1
+ARG VERSION_SHA256=""
+RUN curl -sSL https://github.com/clastix/kamaji/archive/refs/tags/${VERSION}.tar.gz \
+    -o /tmp/kamaji.tar.gz \
+    && echo "${VERSION_SHA256}  /tmp/kamaji.tar.gz" | sha256sum -c - \
+    && tar -xzvf /tmp/kamaji.tar.gz --strip=1 \
+    && rm /tmp/kamaji.tar.gz
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/system/kamaji/images/kamaji/Dockerfile` at line 10, The Dockerfile
currently pipes curl to tar without verifying the archive; modify the RUN that
uses VERSION to instead download the ${VERSION}.tar.gz to a temporary file,
compute its SHA-256 and compare it against a pinned expected digest (add a
CHECKSUM or SHA256_<VERSION> variable alongside VERSION), fail build if the sums
differ, and only then extract (tar -xzvf) the verified file and --strip=1;
ensure the checksum compare is done with a safe comparison (e.g., sha256sum or
shasum -a 256) so tampered archives abort the build.
packages/apps/kubernetes/README.md (1)

15-15: ⚠️ Potential issue | 🟡 Minor

Stale version range in introduction text.

Line 15 still reads "from 1.28 to 1.33" but the support matrix now covers v1.30 to v1.35. This introductory text is not regenerated by cozyvalues-gen (which only updates the Parameters tables), so it needs to be updated manually or in the README template.

📝 Proposed fix
-Users can select the latest patch versions from 1.28 to 1.33.
+Users can select the latest patch versions from 1.30 to 1.35.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/apps/kubernetes/README.md` at line 15, Update the README
introduction text that currently reads "from 1.28 to 1.33" to reflect the
current supported Kubernetes range "v1.30 to v1.35"; locate the string in
packages/apps/kubernetes/README.md (the introductory sentence) and replace the
old range with the new range, ensuring the wording/format (including the "v"
prefix) matches the support matrix.
🧹 Nitpick comments (8)
packages/system/kamaji/images/kamaji/Dockerfile (1)

2-2: Floating base image tags reduce build reproducibility.

Both golang:1.24 (line 2) and gcr.io/distroless/static:nonroot (line 21) are floating tags. Silent patch-level updates to either can change the compiled binary or the shipped runtime layer between two otherwise identical builds.

Pinning both to immutable digests (@sha256:...) is the standard mitigation for reproducible, auditable container builds:

♻️ Proposed fix to pin base images by digest
-FROM golang:1.24 as builder
+FROM golang:1.24@sha256:<digest> AS builder
 ...
-FROM gcr.io/distroless/static:nonroot
+FROM gcr.io/distroless/static:nonroot@sha256:<digest>

Retrieve the current digests with:

docker buildx imagetools inspect golang:1.24 --format '{{json .Manifest.Digest}}'
docker buildx imagetools inspect gcr.io/distroless/static:nonroot --format '{{json .Manifest.Digest}}'

Also applies to: 21-21

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/system/kamaji/images/kamaji/Dockerfile` at line 2, The Dockerfile
uses floating base tags which break reproducibility; pin both base images by
replacing the identifiers "golang:1.24" (the builder stage FROM line) and
"gcr.io/distroless/static:nonroot" (the final stage FROM) with their immutable
digest forms (e.g. golang@sha256:... and
gcr.io/distroless/static@sha256:...)—obtain the correct digests via `docker
buildx imagetools inspect` as suggested and update those two FROM lines to the
corresponding `@sha256` digests so builds are deterministic and auditable.
packages/system/kamaji/charts/kamaji-crds/templates/kamaji.clastix.io_tenantcontrolplanes.yaml (1)

10-10: Use {{- to eliminate the spurious blank line under spec:.

The two leading spaces before {{ are preserved in the output, and nindent 2 prepends a newline, resulting in an empty line between spec: and the first CRD spec key. While valid YAML, using {{- strips the preceding whitespace so nindent 2 alone provides the correct formatting.

This pattern appears in all three CRD templates in this chart and should be updated consistently.

♻️ Proposed fix
 spec:
-  {{ tpl (.Files.Get "hack/kamaji.clastix.io_tenantcontrolplanes_spec.yaml") . | nindent 2 }}
+{{- tpl (.Files.Get "hack/kamaji.clastix.io_tenantcontrolplanes_spec.yaml") . | nindent 2 }}

Apply the same change to kamaji.clastix.io_datastores.yaml and kamaji.clastix.io_kubeconfiggenerators.yaml.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@packages/system/kamaji/charts/kamaji-crds/templates/kamaji.clastix.io_tenantcontrolplanes.yaml`
at line 10, Replace the current tpl invocation that leaves a spurious blank line
after "spec:" by switching the opening action to a whitespace-trimming form;
specifically update the template expression using tpl (.Files.Get
"hack/kamaji.clastix.io_tenantcontrolplanes_spec.yaml") . | nindent 2 to use the
left-trimmed action (change the leading "{{" to "{{-") so nindent 2 no longer
inserts an extra blank line. Apply the same change to the equivalent tpl
invocations in the other two CRD templates (kamaji.clastix.io_datastores.yaml
and kamaji.clastix.io_kubeconfiggenerators.yaml) so all three use the
left-trimmed "{{-" form.
packages/apps/kubernetes/files/versions.yaml (1)

1-6: EOL status of included versions warrants an operator notice.

Kubernetes 1.32 entered maintenance mode on Dec 28, 2025, with an End of Life date of Feb 28, 2026 — this month. Releases 1.31 and older are listed as no longer supported by the upstream project.

The Kubernetes project currently maintains release branches only for the most recent three minor releases (1.35, 1.34, 1.33).

v1.32 will stop receiving upstream security patches in ~9 days; v1.31 and v1.30 are already past upstream EOL. If Cozystack's support policy intentionally extends beyond upstream EOL (e.g., for migration runway), consider adding a deprecation notice in the UI/docs for these versions so operators know to plan upgrades.

Additionally, Kubernetes 1.35 is the last release supporting containerd 1.x. Users must switch to containerd 2.0 or later before upgrading to the next Kubernetes version. This is worth surfacing as a release note for anyone upgrading worker nodes to v1.35.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/apps/kubernetes/files/versions.yaml` around lines 1 - 6, The
versions.yaml list includes v1.32, v1.31 and v1.30 which are at or past upstream
EOL and v1.35 is the last to support containerd 1.x; update the repository so
operators see this: add deprecation/EOL metadata or a linked notice for the keys
"v1.32", "v1.31", and "v1.30" (e.g., a deprecation flag and EOL date) and add a
release note/notice tied to "v1.35" informing users that containerd 1.x is
supported only up to v1.35 and they must migrate to containerd 2.x+ before
upgrading beyond v1.35; ensure this change is surfaced to UI/docs where
versions.yaml is consumed.
packages/system/kamaji/images/kamaji/patches/disable-datastore-check.diff (1)

1-24: Startup DataStore validation removed — confirm operational intent.

The regenerated patch now removes the datastoreutils.CheckExists call entirely (previously it just switched context; now neither import nor call remains). This means Kamaji will no longer fail fast at startup if the configured DataStore is unavailable. The file name makes the intent clear, but ensure monitoring/alerting compensates for the removed fail-fast guard.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/system/kamaji/images/kamaji/patches/disable-datastore-check.diff`
around lines 1 - 24, The startup DataStore existence check was removed
(datastoreutils.CheckExists) causing Kamaji to no longer fail-fast if the
configured DataStore is unavailable; restore the original behavior by re-adding
the datastoreutils import and calling datastoreutils.CheckExists(ctx, scheme,
datastore) inside NewCmd (before the controllerReconcileTimeout validation),
returning the error if it fails; ensure the import alias matches datastoreutils
and the error handling returns the error up the stack so startup fails fast when
the DataStore is unreachable.
packages/system/kamaji/charts/kamaji/Chart.yaml (1)

2-2: appVersion: latest — verify intentional upstream versioning strategy.

Using latest as appVersion diverges from semver conventions (and from the previous v0.0.0). Given the ArtifactHub annotation at line 50 (Releasing latest chart at every push), this appears intentional for a rolling-latest track. Confirm this matches the intended upgrade path and that any downstream tooling (e.g., image-pinning logic) handles a non-semver appVersion correctly.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/system/kamaji/charts/kamaji/Chart.yaml` at line 2, The Chart.yaml
currently sets appVersion: latest which breaks semver expectations; confirm
whether the rolling-latest release strategy is intentional and either restore a
semver value (e.g., vX.Y.Z) in appVersion or add clear documentation/annotations
and update downstream tooling (image-pinning, upgrade checks, ArtifactHub
integrations) to accept non-semver values; check and update any code paths that
read Chart.yaml.appVersion (search for references to appVersion in Chart.yaml,
the ArtifactHub annotation, image-pinning logic, and release scripts) to ensure
they correctly handle the "latest" string or change it back to a proper semver.
packages/system/kamaji/charts/kamaji/values.yaml (1)

115-158: Add configurable livenessProbe and readinessProbe fields to kubeconfigGenerator for pattern consistency.

The main Kamaji controller exposes livenessProbe and readinessProbe as user-configurable top-level values (lines 26–39), but the kubeconfigGenerator block lacks these fields. While the deployment template only sets healthProbeBindAddress without probe configuration, adding explicit probe values would align with the established pattern and allow users to customize probe behavior if needed.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/system/kamaji/charts/kamaji/values.yaml` around lines 115 - 158, Add
configurable livenessProbe and readinessProbe fields under the
kubeconfigGenerator values block so users can override probe behavior (similar
to the top-level livenessProbe/readinessProbe). Specifically, in the
kubeconfigGenerator section (which contains healthProbeBindAddress,
replicaCount, serviceAccountOverride, etc.) add livenessProbe: {} and
readinessProbe: {} with the same configurable shape used by the main controller,
ensuring the deployment template consumes these new keys instead of only
healthProbeBindAddress; update any references in the Kubeconfig Generator
deployment manifest to use kubeconfigGenerator.livenessProbe and
kubeconfigGenerator.readinessProbe.
packages/system/kamaji/charts/kamaji/templates/kubeconfiggenerator-deployment.yaml (1)

27-41: Missing liveness and readiness probes despite healthProbeBindAddress being configured.

The container has --health-probe-bind-address configured but no livenessProbe or readinessProbe are defined, unlike the main Kamaji controller. Without probes, Kubernetes cannot detect an unhealthy container and will not restart or route traffic accordingly.

♻️ Suggested addition (align with main Kamaji controller's probe config)
           resources:
             {{- toYaml .Values.kubeconfigGenerator.resources | nindent 12 }}
           securityContext:
             {{- toYaml .Values.kubeconfigGenerator.securityContext | nindent 12 }}
+          livenessProbe:
+            httpGet:
+              path: /healthz
+              port: {{ .Values.kubeconfigGenerator.healthProbeBindAddress | trimPrefix ":" }}
+            initialDelaySeconds: 15
+            periodSeconds: 20
+          readinessProbe:
+            httpGet:
+              path: /readyz
+              port: {{ .Values.kubeconfigGenerator.healthProbeBindAddress | trimPrefix ":" }}
+            initialDelaySeconds: 5
+            periodSeconds: 10
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In
`@packages/system/kamaji/charts/kamaji/templates/kubeconfiggenerator-deployment.yaml`
around lines 27 - 41, Add livenessProbe and readinessProbe to the
kubeconfig-generator container spec (the container named "controller") so
Kubernetes can detect unhealthy instances; use HTTP probes that hit the health
endpoints (e.g., /healthz for liveness and /readyz for readiness) on the same
health port/address used by the --health-probe-bind-address flag (mirror the
main Kamaji controller probe configuration: set initialDelaySeconds,
periodSeconds, timeoutSeconds, failureThreshold/successThreshold), and place
these probe blocks alongside the existing resources and securityContext in the
container definition, wired to the chart values (e.g.,
.Values.kubeconfigGenerator.healthProbeBindAddress /
.Values.kubeconfigGenerator.healthPort or default port) so they respect
user-configured probe settings.
packages/system/kamaji/charts/kamaji-crds/templates/_helpers.tpl (1)

13-24: Rename kamaji.fullname to kamaji-crds.fullname for naming consistency.

This template defines kamaji.fullname but it is never used in the chart. The definition breaks the kamaji-crds.* naming convention that all other helpers follow (kamaji-crds.name, kamaji-crds.chart, kamaji-crds.certManagerAnnotation, kamaji-crds.labels). Either rename it to match the convention or remove it if it's truly unused.

♻️ Suggested rename
-{{- define "kamaji.fullname" -}}
+{{- define "kamaji-crds.fullname" -}}
 {{- if .Values.fullnameOverride }}
 {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
 {{- else }}
 {{- $name := default .Chart.Name .Values.nameOverride }}
 {{- if contains $name .Release.Name }}
 {{- .Release.Name | trunc 63 | trimSuffix "-" }}
 {{- else }}
 {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
 {{- end }}
 {{- end }}
 {{- end }}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/system/kamaji/charts/kamaji-crds/templates/_helpers.tpl` around
lines 13 - 24, The helper currently defines "kamaji.fullname" which breaks the
chart's "kamaji-crds.*" helper naming convention; rename the template definition
from "kamaji.fullname" to "kamaji-crds.fullname" (or delete it if truly unused)
and update any internal references to use "kamaji-crds.fullname" so it matches
the existing helpers like "kamaji-crds.name", "kamaji-crds.chart", and
"kamaji-crds.labels"; ensure the definition body (logic using
.Values.fullnameOverride, $name, .Release.Name, trunc/trimSuffix) remains
identical after the rename.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/apps/kubernetes/files/versions.yaml`:
- Line 1: The entry "v1.35": "v1.35.0" in versions.yaml is stale; re-run the
version update script (update-versions.sh) to regenerate versions.yaml so the
value becomes "v1.35.1". Run the script used to populate versions.yaml, commit
the regenerated file, and verify the updated key/value "v1.35": "v1.35.1" is
present.

In `@packages/system/kamaji/charts/kamaji-crds/Chart.yaml`:
- Around line 19-29: The artifacthub.io/crds annotation in Chart.yaml is missing
the KubeconfigGenerator CRD entry; update the annotation block that currently
lists TenantControlPlane and DataStore to also include a KubeconfigGenerator
entry (matching the other CRDs' format) so artifact hub can surface it—add a
list item with kind: KubeconfigGenerator, version: v1alpha1, name:
kubeconfiggenerators.kamaji.clastix.io, displayName: KubeconfigGenerator and a
brief description consistent with the existing entries.

In `@packages/system/kamaji/charts/kamaji-crds/Makefile`:
- Around line 1-9: Add a .PHONY declaration for the Makefile targets so make
always runs them: update the Makefile (targets "docs" and "docker") to include a
line like ".PHONY: docs docker" near the top so neither "docs" nor "docker" will
be treated as up-to-date files if a same-named artifact exists.

In `@packages/system/kamaji/charts/kamaji/templates/_helpers.tpl`:
- Around line 97-102: The template helper kamaji.kubeconfigGeneratorName
incorrectly uses .Values.fullnameOverride in the override branch; update the
override branch to use .Values.kubeconfigGenerator.fullnameOverride (the same
key checked in the if) so the generated name comes from
kubeconfigGenerator.fullnameOverride; ensure you still apply trunc 63 and
trimSuffix "-" to that value just like the else branch.

In
`@packages/system/kamaji/charts/kamaji/templates/kubeconfiggenerator-deployment.yaml`:
- Line 25: The serviceAccount override is never used because the Helm default
arguments are reversed; change the default call so the included name is the
first arg and the override (.Values.kubeconfigGenerator.serviceAccountOverride)
is the second so the override is returned when non-empty (i.e., replace default
.Values.kubeconfigGenerator.serviceAccountOverride (include
"kamaji.serviceAccountName" .) with default (include "kamaji.serviceAccountName"
.) .Values.kubeconfigGenerator.serviceAccountOverride to ensure
serviceAccountName honors the override).
- Around line 11-21: The selector collision is caused by both Deployments
calling the kamaji.selectorLabels helper with the same context, so update the
kubeconfigGenerator Deployment to call include "kamaji.selectorLabels" with an
overridden component (e.g., pass a dict with "component":"kubeconfig-generator")
for both the selector.matchLabels and the pod template labels; specifically
change the two occurrences of include "kamaji.selectorLabels" . to include
"kamaji.selectorLabels" (dict "component" "kubeconfig-generator" "Release" .) or
an equivalent dict-based context so the rendered app.kubernetes.io/component
becomes kubeconfig-generator and prevents ReplicaSet/pod collisions.

---

Outside diff comments:
In `@packages/apps/kubernetes/README.md`:
- Line 15: Update the README introduction text that currently reads "from 1.28
to 1.33" to reflect the current supported Kubernetes range "v1.30 to v1.35";
locate the string in packages/apps/kubernetes/README.md (the introductory
sentence) and replace the old range with the new range, ensuring the
wording/format (including the "v" prefix) matches the support matrix.

In `@packages/system/kamaji/images/kamaji/Dockerfile`:
- Line 10: The Dockerfile currently pipes curl to tar without verifying the
archive; modify the RUN that uses VERSION to instead download the
${VERSION}.tar.gz to a temporary file, compute its SHA-256 and compare it
against a pinned expected digest (add a CHECKSUM or SHA256_<VERSION> variable
alongside VERSION), fail build if the sums differ, and only then extract (tar
-xzvf) the verified file and --strip=1; ensure the checksum compare is done with
a safe comparison (e.g., sha256sum or shasum -a 256) so tampered archives abort
the build.

---

Nitpick comments:
In `@packages/apps/kubernetes/files/versions.yaml`:
- Around line 1-6: The versions.yaml list includes v1.32, v1.31 and v1.30 which
are at or past upstream EOL and v1.35 is the last to support containerd 1.x;
update the repository so operators see this: add deprecation/EOL metadata or a
linked notice for the keys "v1.32", "v1.31", and "v1.30" (e.g., a deprecation
flag and EOL date) and add a release note/notice tied to "v1.35" informing users
that containerd 1.x is supported only up to v1.35 and they must migrate to
containerd 2.x+ before upgrading beyond v1.35; ensure this change is surfaced to
UI/docs where versions.yaml is consumed.

In `@packages/system/kamaji/charts/kamaji-crds/templates/_helpers.tpl`:
- Around line 13-24: The helper currently defines "kamaji.fullname" which breaks
the chart's "kamaji-crds.*" helper naming convention; rename the template
definition from "kamaji.fullname" to "kamaji-crds.fullname" (or delete it if
truly unused) and update any internal references to use "kamaji-crds.fullname"
so it matches the existing helpers like "kamaji-crds.name", "kamaji-crds.chart",
and "kamaji-crds.labels"; ensure the definition body (logic using
.Values.fullnameOverride, $name, .Release.Name, trunc/trimSuffix) remains
identical after the rename.

In
`@packages/system/kamaji/charts/kamaji-crds/templates/kamaji.clastix.io_tenantcontrolplanes.yaml`:
- Line 10: Replace the current tpl invocation that leaves a spurious blank line
after "spec:" by switching the opening action to a whitespace-trimming form;
specifically update the template expression using tpl (.Files.Get
"hack/kamaji.clastix.io_tenantcontrolplanes_spec.yaml") . | nindent 2 to use the
left-trimmed action (change the leading "{{" to "{{-") so nindent 2 no longer
inserts an extra blank line. Apply the same change to the equivalent tpl
invocations in the other two CRD templates (kamaji.clastix.io_datastores.yaml
and kamaji.clastix.io_kubeconfiggenerators.yaml) so all three use the
left-trimmed "{{-" form.

In `@packages/system/kamaji/charts/kamaji/Chart.yaml`:
- Line 2: The Chart.yaml currently sets appVersion: latest which breaks semver
expectations; confirm whether the rolling-latest release strategy is intentional
and either restore a semver value (e.g., vX.Y.Z) in appVersion or add clear
documentation/annotations and update downstream tooling (image-pinning, upgrade
checks, ArtifactHub integrations) to accept non-semver values; check and update
any code paths that read Chart.yaml.appVersion (search for references to
appVersion in Chart.yaml, the ArtifactHub annotation, image-pinning logic, and
release scripts) to ensure they correctly handle the "latest" string or change
it back to a proper semver.

In
`@packages/system/kamaji/charts/kamaji/templates/kubeconfiggenerator-deployment.yaml`:
- Around line 27-41: Add livenessProbe and readinessProbe to the
kubeconfig-generator container spec (the container named "controller") so
Kubernetes can detect unhealthy instances; use HTTP probes that hit the health
endpoints (e.g., /healthz for liveness and /readyz for readiness) on the same
health port/address used by the --health-probe-bind-address flag (mirror the
main Kamaji controller probe configuration: set initialDelaySeconds,
periodSeconds, timeoutSeconds, failureThreshold/successThreshold), and place
these probe blocks alongside the existing resources and securityContext in the
container definition, wired to the chart values (e.g.,
.Values.kubeconfigGenerator.healthProbeBindAddress /
.Values.kubeconfigGenerator.healthPort or default port) so they respect
user-configured probe settings.

In `@packages/system/kamaji/charts/kamaji/values.yaml`:
- Around line 115-158: Add configurable livenessProbe and readinessProbe fields
under the kubeconfigGenerator values block so users can override probe behavior
(similar to the top-level livenessProbe/readinessProbe). Specifically, in the
kubeconfigGenerator section (which contains healthProbeBindAddress,
replicaCount, serviceAccountOverride, etc.) add livenessProbe: {} and
readinessProbe: {} with the same configurable shape used by the main controller,
ensuring the deployment template consumes these new keys instead of only
healthProbeBindAddress; update any references in the Kubeconfig Generator
deployment manifest to use kubeconfigGenerator.livenessProbe and
kubeconfigGenerator.readinessProbe.

In `@packages/system/kamaji/images/kamaji/Dockerfile`:
- Line 2: The Dockerfile uses floating base tags which break reproducibility;
pin both base images by replacing the identifiers "golang:1.24" (the builder
stage FROM line) and "gcr.io/distroless/static:nonroot" (the final stage FROM)
with their immutable digest forms (e.g. golang@sha256:... and
gcr.io/distroless/static@sha256:...)—obtain the correct digests via `docker
buildx imagetools inspect` as suggested and update those two FROM lines to the
corresponding `@sha256` digests so builds are deterministic and auditable.

In `@packages/system/kamaji/images/kamaji/patches/disable-datastore-check.diff`:
- Around line 1-24: The startup DataStore existence check was removed
(datastoreutils.CheckExists) causing Kamaji to no longer fail-fast if the
configured DataStore is unavailable; restore the original behavior by re-adding
the datastoreutils import and calling datastoreutils.CheckExists(ctx, scheme,
datastore) inside NewCmd (before the controllerReconcileTimeout validation),
returning the error if it fails; ensure the import alias matches datastoreutils
and the error handling returns the error up the stack so startup fails fast when
the DataStore is unreachable.

Comment thread packages/apps/kubernetes/files/versions.yaml Outdated
Comment thread packages/system/kamaji/charts/kamaji-crds/Chart.yaml
Comment thread packages/system/kamaji/charts/kamaji-crds/Makefile
Comment thread packages/system/kamaji/charts/kamaji/templates/_helpers.tpl

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/system/kamaji/images/kamaji/Dockerfile (1)

2-2: Consider pinning to a specific Go patch version for reproducible builds.

golang:1.26 is a floating tag. Once 1.26.1 is published, the tag resolves to a different layer, silently changing the build toolchain. Pinning to golang:1.26.0 (or a digest) prevents inadvertent toolchain drift between builds.

♻️ Proposed fix
-FROM golang:1.26 AS builder
+FROM golang:1.26.0 AS builder
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/system/kamaji/images/kamaji/Dockerfile` at line 2, The Dockerfile's
FROM instruction uses a floating Go tag ("FROM golang:1.26 AS builder"); change
it to a pinned patch version (e.g., "golang:1.26.0") or a digest to ensure
reproducible builds by preventing toolchain drift—update the FROM line in the
Dockerfile accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@packages/system/kamaji/images/kamaji/Dockerfile`:
- Line 2: The Dockerfile's FROM instruction uses a floating Go tag ("FROM
golang:1.26 AS builder"); change it to a pinned patch version (e.g.,
"golang:1.26.0") or a digest to ensure reproducible builds by preventing
toolchain drift—update the FROM line in the Dockerfile accordingly.

…mport

Kamaji edge-26.2.4 requires Go >= 1.25.0, update base image accordingly.
Also remove unused "context" import from disable-datastore-check patch,
since removing the CheckExists call was the only usage of that package.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
CAPI Kamaji provider v0.15.0 is incompatible with Kamaji edge-26.2.4
due to the new dataStoreUsername field with XValidation rule. The
provider's CreateOrUpdate drops the field (not in its Go types),
triggering "unsetting the dataStoreUsername is not supported" error.

This results in KamajiControlPlane staying INITIALIZED=false even
though the underlying TenantControlPlane reaches Ready.

v0.16.0 includes support for DataStoreUsername (PR #243 in v0.15.4)
and updated Kamaji types compatible with edge-26.2.4.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
@dosubot dosubot Bot added size/XL This PR changes 500-999 lines, ignoring generated files and removed size/XXL This PR changes 1000+ lines, ignoring generated files labels Feb 19, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/system/kamaji/images/kamaji/Dockerfile (1)

2-2: Consider pinning to a specific patch-version tag for reproducible builds

golang:1.25 is a floating minor-version tag that will silently resolve to different patch images over time, making builds non-reproducible. Pinning to a specific patch (e.g., golang:1.25.7) or an image digest ensures the same toolchain is used on every build.

♻️ Proposed fix (pin to specific patch version)
-FROM golang:1.25 AS builder
+FROM golang:1.25.7 AS builder

Or, for maximum reproducibility, use a digest:

-FROM golang:1.25 AS builder
+FROM golang:1.25.7@sha256:<digest> AS builder
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/system/kamaji/images/kamaji/Dockerfile` at line 2, Update the Docker
base image specification to pin the floating tag "FROM golang:1.25" to a
specific patch version or digest so builds are reproducible; replace the
floating tag with a concrete patched tag like "golang:1.25.7" or use an image
digest (sha256) in the same FROM instruction to ensure the same toolchain is
used for every build.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/system/kamaji/images/kamaji/Dockerfile`:
- Line 2: Replace the floating Go base image tag in the Dockerfile (the FROM
instruction using golang:1.25) with a specific patch-pinned tag (for example
golang:1.25.7) to ensure reproducible builds; update the FROM line in
packages/system/kamaji/images/kamaji/Dockerfile and confirm the exact patch
version with the PR author to match Kamaji edge-26.2.4 requirements.

---

Nitpick comments:
In `@packages/system/kamaji/images/kamaji/Dockerfile`:
- Line 2: Update the Docker base image specification to pin the floating tag
"FROM golang:1.25" to a specific patch version or digest so builds are
reproducible; replace the floating tag with a concrete patched tag like
"golang:1.25.7" or use an image digest (sha256) in the same FROM instruction to
ensure the same toolchain is used for every build.

@@ -1,7 +1,7 @@
# Build the manager binary
FROM golang:1.24 as builder
FROM golang:1.25 AS builder

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

# Check the actual Dockerfile content
cat packages/system/kamaji/images/kamaji/Dockerfile | head -10

Repository: cozystack/cozystack

Length of output: 298


🏁 Script executed:

# Find the git commit message for this PR to verify the stated intent
git log --oneline -20 | head -20

Repository: cozystack/cozystack

Length of output: 130


🏁 Script executed:

# Also check if there's any documentation about why golang:1.25 was chosen
rg -i "golang.*1\.25|golang.*1\.26" packages/system/kamaji --type-list

Repository: cozystack/cozystack

Length of output: 452


🏁 Script executed:

# Search for commits that modified the Kamaji Dockerfile
git log --oneline -- packages/system/kamaji/images/kamaji/Dockerfile | head -10

Repository: cozystack/cozystack

Length of output: 130


🏁 Script executed:

# Search for any references to golang version changes or Kamaji version updates in recent commits
git log --oneline --all | grep -i "kamaji\|golang" | head -20

Repository: cozystack/cozystack

Length of output: 1187


🏁 Script executed:

# Check if there are any comments or documentation about the golang version choice
rg "golang" packages/system/kamaji --type dockerfile -A 2 -B 2

Repository: cozystack/cozystack

Length of output: 97


🏁 Script executed:

# Check the full git history of the Dockerfile to see previous golang versions
git log -p -- packages/system/kamaji/images/kamaji/Dockerfile | grep -A 5 -B 5 "golang:" | head -50

Repository: cozystack/cozystack

Length of output: 45


🏁 Script executed:

# Search for any Kamaji version or golang version requirements in the codebase
rg "edge-26\.2\.4" packages/system/kamaji

Repository: cozystack/cozystack

Length of output: 134


🏁 Script executed:

# Check if there are any build or requirements documentation
find packages/system/kamaji -name "*.md" -o -name "*.txt" -o -name "go.mod" | head -20

Repository: cozystack/cozystack

Length of output: 270


🏁 Script executed:

# Check if Kamaji repository or any documentation mentions golang version requirements
rg "golang|Go.*version" packages/system/kamaji/charts/kamaji/README.md

Repository: cozystack/cozystack

Length of output: 45


🏁 Script executed:

# Search the Kamaji tarball source extraction for any golang version requirements
# First, let's look at what gets extracted from the Kamaji release
rg "go.mod|golang" packages/system/kamaji --type-list 2>/dev/null || find packages/system/kamaji -name "go.mod"

Repository: cozystack/cozystack

Length of output: 45


🏁 Script executed:

# Verify golang docker image tags using a web search approach - check if golang:1.26 exists
curl -s "https://registry.hub.docker.com/v2/repositories/library/golang/tags/?page_size=100" | python3 << 'EOF'
import sys, json
try:
    data = json.load(sys.stdin)
    results = data.get('results', [])
    golang_versions = [t['name'] for t in results if t['name'] in ['1.25', '1.26', '1.25.7', '1.26.0']]
    print("Found golang Docker tags:", golang_versions)
except Exception as e:
    print(f"Error: {e}")
EOF

Repository: cozystack/cozystack

Length of output: 111


Pin golang base image to specific patch version for reproducibility

The Dockerfile uses golang:1.25, a floating minor-version tag that resolves to the latest 1.25.x patch at build time, preventing reproducible builds. Consider pinning to a specific patch version (e.g., golang:1.25.7) to ensure consistent builds across environments.

Additionally, verify the intended golang version with the PR author. The current value golang:1.25 satisfies Kamaji edge-26.2.4's requirements, but clarification on the golang version choice would be helpful.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/system/kamaji/images/kamaji/Dockerfile` at line 2, Replace the
floating Go base image tag in the Dockerfile (the FROM instruction using
golang:1.25) with a specific patch-pinned tag (for example golang:1.25.7) to
ensure reproducible builds; update the FROM line in
packages/system/kamaji/images/kamaji/Dockerfile and confirm the exact patch
version with the PR author to match Kamaji edge-26.2.4 requirements.

Go 1.26 was released on 2026-02-10 and is fully compatible with
Kamaji edge-26.2.4 (which requires go 1.25.0 in go.mod).
Verified by local build.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
v1.35.1 was released on 2026-02-10.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
Kamaji auto-derives the konnectivity proxy image tag as v0.{minor}.0
from the Kubernetes version. For K8s v1.35, this produces v0.35.0,
but the kas-network-proxy/proxy-server:v0.35.0 image does not exist
in registry.k8s.io yet, causing ImagePullBackOff on new TCP pods.

Add konnectivity-versions.yaml mapping to explicitly override the
konnectivity version when the auto-derived tag is unavailable.
For v1.35, pin to v0.34.0 (latest available).

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
When running kubernetes-latest and kubernetes-previous E2E tests
simultaneously, worker VMs compete for resources in the sandbox
environment. 3 minutes was insufficient for nodes to boot and
join the tenant cluster under load. Increase to 5 minutes.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
hack/e2e-apps/run-kubernetes.sh (2)

74-75: ⚠️ Potential issue | 🟡 Minor

Stale comment — says "2 minutes" but the actual timeout is 5m.

🔧 Proposed fix
-  # Wait for Kubernetes resources to be ready (timeout after 2 minutes)
+  # Wait for Kubernetes resources to be ready (timeout after 5 minutes)
   kubectl wait tcp -n tenant-test kubernetes-${test_name} --timeout=5m --for=jsonpath='{.status.kubernetesResources.version.status}'=Ready
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@hack/e2e-apps/run-kubernetes.sh` around lines 74 - 75, Update the stale
inline comment above the kubectl wait invocation: change the comment string "#
Wait for Kubernetes resources to be ready (timeout after 2 minutes)" to reflect
the actual timeout used by the kubectl command (5 minutes). The relevant snippet
to update is the comment immediately preceding the kubectl wait tcp -n
tenant-test kubernetes-${test_name} --timeout=5m
--for=jsonpath='{.status.kubernetesResources.version.status}'=Ready so the
comment text and the --timeout value are consistent.

212-221: ⚠️ Potential issue | 🟡 Minor

Pre-existing bug: curl succeeding on the 20th attempt is misreported as failure.

After break, $i is still 20, so [ "$i" -eq 20 ] evaluates to true even on a successful last-attempt hit. A simple fix is to track success with a flag, or to use $? / a sentinel instead.

🐛 Proposed fix
+  lb_ok=false
   for i in $(seq 1 20); do
     echo "Attempt $i"
-    curl --silent --fail "http://${LB_ADDR}" && break
+    if curl --silent --fail "http://${LB_ADDR}"; then
+      lb_ok=true
+      break
+    fi
     sleep 3
   done

-  if [ "$i" -eq 20 ]; then
+  if [ "$lb_ok" != "true" ]; then
     echo "LoadBalancer not reachable" >&2
     exit 1
   fi
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@hack/e2e-apps/run-kubernetes.sh` around lines 212 - 221, The loop that polls
the load balancer uses the loop index i to determine failure, but i remains 20
even when curl succeeds on the last iteration, causing a false failure; update
the script to track success with a sentinel (e.g., initialize LB_REACHABLE=0
before the for-loop, set LB_REACHABLE=1 when curl --silent --fail succeeds and
then break) and after the loop test the sentinel (LB_REACHABLE) instead of
comparing i to 20 so a success on the final attempt is reported correctly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@hack/e2e-apps/run-kubernetes.sh`:
- Around line 74-75: Update the stale inline comment above the kubectl wait
invocation: change the comment string "# Wait for Kubernetes resources to be
ready (timeout after 2 minutes)" to reflect the actual timeout used by the
kubectl command (5 minutes). The relevant snippet to update is the comment
immediately preceding the kubectl wait tcp -n tenant-test
kubernetes-${test_name} --timeout=5m
--for=jsonpath='{.status.kubernetesResources.version.status}'=Ready so the
comment text and the --timeout value are consistent.
- Around line 212-221: The loop that polls the load balancer uses the loop index
i to determine failure, but i remains 20 even when curl succeeds on the last
iteration, causing a false failure; update the script to track success with a
sentinel (e.g., initialize LB_REACHABLE=0 before the for-loop, set
LB_REACHABLE=1 when curl --silent --fail succeeds and then break) and after the
loop test the sentinel (LB_REACHABLE) instead of comparing i to 20 so a success
on the final attempt is reported correctly.

- Kill stale port-forward processes before starting a new one;
  on retries, the previous attempt's port-forward still holds the
  port, causing all kubectl commands to get "connection refused"
- Use -ge 2 instead of -eq 2 for node count check; MachineHealthCheck
  may create a 3rd VM, leading to 3 nodes joining the tenant cluster
  which would never satisfy the exact equality check
- Increase node join timeout from 5m to 8m; QEMU VMs with v1.34 need
  more time to boot and join when running after kubernetes-latest

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Feb 20, 2026
Kamaji edge-26.2.4 is compiled against Kubernetes 1.35 libraries.
SetDefaults_KubeletConfiguration() from 1.35 injects two fields
gated by feature gates that are not enabled in earlier versions:
- crashLoopBackOff.maxContainerRestartPeriod (KubeletCrashLoopBackOffMax)
- imagePullCredentialsVerificationPolicy (KubeletEnsureSecretPulledImages)

Kubelets < 1.35 reject these fields during configuration validation,
causing worker nodes to fail to join the tenant cluster.

Add a Go patch that clears these fields from the kubelet-config
ConfigMap when the target Kubernetes version is below 1.35.

See: clastix/kamaji#1062

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aleksei Sviridkin <f@lex.la>
@kvaps Andrei Kvapil (kvaps) merged commit d856775 into main Feb 20, 2026
6 checks passed
@kvaps Andrei Kvapil (kvaps) deleted the feat/update-kubernetes-1.35 branch February 20, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/feature Categorizes issue or PR as related to a new feature lgtm This PR has been approved by a maintainer size/XL This PR changes 500-999 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants