Skip to content

fix file non-existent error when kmesh start with --enable-ipsec=true#1473

Merged
kmesh-bot merged 4 commits intokmesh-net:mainfrom
zrggw:start_with_ipsec
Aug 12, 2025
Merged

fix file non-existent error when kmesh start with --enable-ipsec=true#1473
kmesh-bot merged 4 commits intokmesh-net:mainfrom
zrggw:start_with_ipsec

Conversation

@zrggw
Copy link
Copy Markdown
Contributor

@zrggw zrggw commented Aug 6, 2025

What type of PR is this?
/kind bug
/kind enhancement

What this PR does / why we need it:
When kmesh starts, if you set --enable-ipsec=true, it will report errors during initialization of the ipsec controller because it reads non-existent file. Add a check for whether the file exists before reading the file.

Which issue(s) this PR fixes:

Fixes #1289

Special notes for your reviewer:

Does this PR introduce a user-facing change?:


…psec=true, so resolve no such file error

Signed-off-by: aicee <hhbin2000@foxmail.com>
Copilot AI review requested due to automatic review settings August 6, 2025 10:01
@kmesh-bot kmesh-bot added the kind/bug Something isn't working label Aug 6, 2025
@kmesh-bot kmesh-bot requested review from YaoZengzeng and nlgwcy August 6, 2025 10:01
@kmesh-bot kmesh-bot added kind/enhancement New feature or request size/L labels Aug 6, 2025
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @zrggw, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical initialization error in Kmesh when IPsec is enabled. Previously, the IPsec controller would fail to start because it attempted to read the IPsec key secret before it was created. My changes introduce a mechanism to automatically generate and persist a default IPsec key as a Kubernetes Secret if one is not found, ensuring the controller can always initialize successfully. This involves updates to the IPsec handler, controller logic, Kubernetes deployment configurations (DaemonSet and RBAC), and includes new unit tests to validate the key generation and secret management.

Highlights

  • Automatic IPsec Key Generation: I've introduced a new initConfig function within the IPsec handler that automatically generates a default IPsec key and stores it as a Kubernetes Secret if one doesn't already exist. This ensures that the IPsec controller has a key available from the start.
  • Improved IPsec Controller Initialization: The IPsec controller's initialization flow has been updated to call this new initConfig function. This resolves an issue where the controller would fail if it tried to read the IPsec key secret before it was created.
  • Kubernetes Deployment Configuration Updates: I've updated the Kmesh Helm chart's daemonset.yaml to include a volume mount for the IPsec key file and to define the kmesh-ipsec secret volume, making the key accessible to the Kmesh pods.
  • Enhanced RBAC Permissions: The necessary RBAC permissions have been added to kmesh-rbac.yaml and clusterrole.yaml, allowing Kmesh to get, create, update, delete, list, and watch Kubernetes secrets resources, which is essential for managing the IPsec key.
  • New Unit Tests for Key Management: Comprehensive unit tests have been added for the initConfig function, verifying its behavior in scenarios where the secret already exists (ensuring it's not overwritten) and confirming the randomness of generated keys.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.


A secret key, unseen, Generated, strong, and keen. If not found, then made anew, For secure paths, fresh and true.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR addresses a bug where Kmesh fails to initialize when IPsec is enabled due to a missing IPsec key secret. The solution generates a default IPsec key during controller initialization to prevent startup failures.

  • Adds an initConfig function that creates a default IPsec key secret if none exists
  • Updates error handling to log warnings instead of failing initialization when key files are missing
  • Adds RBAC permissions for secret management and volume mounts for IPsec configuration

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
pkg/controller/encryption/ipsec/ipsec_handler.go Implements initConfig function to generate default IPsec keys and updates constants
pkg/controller/encryption/ipsec/ipsec_controller.go Calls initConfig during initialization and changes file loading errors to warnings
pkg/controller/encryption/ipsec/ipsec_handler_test.go Adds comprehensive unit tests for the new initConfig functionality
deploy/yaml/clusterrole.yaml Grants secrets management permissions to the cluster role
deploy/charts/kmesh-helm/templates/kmesh-rbac.yaml Grants secrets management permissions in Helm chart RBAC
deploy/charts/kmesh-helm/templates/daemonset.yaml Adds optional secret volume mount for IPsec configuration
Comments suppressed due to low confidence (1)

pkg/controller/encryption/ipsec/ipsec_handler_test.go:673

  • [nitpick] This test only runs 5 iterations to check key uniqueness. With a 32-byte key, the probability of collision is extremely low, but the test could be strengthened by verifying that keys are not all zeros or have reasonable entropy characteristics.
		assert.False(t, aeadKeySet[string(key.AeadKey)], "Key should be unique for each initConfig call")

@zrggw
Copy link
Copy Markdown
Contributor Author

zrggw commented Aug 6, 2025

cc @hzxuzhonghu

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses a startup issue with the IPsec controller by ensuring the IPsec key secret is created before being used. The changes are logically sound, but there are a few critical issues to address. Specifically, a potential security vulnerability due to an unchecked error in random key generation, and error handling that could lead to silent failures of the IPsec functionality. I've provided detailed comments and suggestions to improve the robustness and security of the implementation.

@LiZhenCheng9527
Copy link
Copy Markdown
Contributor

In my opinion, We shoudld add a create sub command in kmeshctl secret or add kmeshctl ipsec create to create a secret for IPsec. Such command return the created secret namespacedName. Choose between automatic creation or manual creation for user.

@hzxuzhonghu
Copy link
Copy Markdown
Member

Added an initConfig function in the ipsec handler.

The initConfig function generates a default IPsecKey when initializing the controller.
In the NewIPsecController function, loading this file may fail, but it will still be read later in the StartWatch function.

I am not sure why not waiting until the secret mounted rather than generating an useless key

@zrggw
Copy link
Copy Markdown
Contributor Author

zrggw commented Aug 7, 2025

Added an initConfig function in the ipsec handler.

The initConfig function generates a default IPsecKey when initializing the controller.
In the NewIPsecController function, loading this file may fail, but it will still be read later in the StartWatch function.

I am not sure why not waiting until the secret mounted rather than generating an useless key

To avoid an error (#1289) during IPSec controller initialization, the ./kmesh-ipsec/ipsec file (which contains the necessary keys like SPI, aeadKeyName, aeadKey, etc.) must be present. If you did not use kmeshctl secret to generate this file before starting kmesh, althrough secret kmesh-ipsec will be mounted, but secret kmesh-ipsec secret hasn't been created but ./kmesh-ipsec/ipsec is missing, so using initConfig to create secret kmesh-ipsec which has ipSec in Secret.Data(so ./kemsh/kmesh-ipsec/ipSec file will be created and can be readed).

@zrggw
Copy link
Copy Markdown
Contributor Author

zrggw commented Aug 7, 2025

I understand, so is there no need to add IPsec configuration when enabling --enable-ipsec=true during kmesh startup? And only let kmeshctl to mange ipsec secret? @hzxuzhonghu

@zrggw zrggw changed the title generate ipsec key when kmesh start with --enable-ipsec=true fix file non-existent error when kmesh start with --enable-ipsec=true Aug 8, 2025
zrggw added 2 commits August 8, 2025 09:50
Signed-off-by: aicee <hhbin2000@foxmail.com>
Signed-off-by: aicee <hhbin2000@foxmail.com>
@kmesh-bot kmesh-bot added size/S and removed size/L labels Aug 8, 2025
@zrggw
Copy link
Copy Markdown
Contributor Author

zrggw commented Aug 8, 2025

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly addresses a startup error when IPsec is enabled without the necessary secret by making the secret volume mount optional and checking for the key file's existence before use. The changes are logical and well-implemented. My review includes one suggestion to improve the robustness of the file existence check in ipsec_controller.go to ensure that potential configuration or permission errors are logged, rather than being silently ignored, which is important for a security-related feature.

@codecov
Copy link
Copy Markdown

codecov bot commented Aug 8, 2025

Codecov Report

❌ Patch coverage is 37.50000% with 5 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@af3e8e5). Learn more about missing BASE report.
⚠️ Report is 32 commits behind head on main.

Files with missing lines Patch % Lines
...kg/controller/encryption/ipsec/ipsec_controller.go 28.57% 4 Missing and 1 partial ⚠️
Files with missing lines Coverage Δ
pkg/controller/encryption/ipsec/ipsec_handler.go 42.06% <100.00%> (ø)
...kg/controller/encryption/ipsec/ipsec_controller.go 47.66% <28.57%> (ø)

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update af3e8e5...89a29e7. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: aicee <hhbin2000@foxmail.com>
@zrggw
Copy link
Copy Markdown
Contributor Author

zrggw commented Aug 8, 2025

I was thinking maybe we can optimize the secret part of kmeshctl related to IPsec and design the command as kmeshctl secret [command] [flags].

  1. kmeshctl secret create // Create a secret configuration for IPsec, using a random key
  2. kmeshctl secret create --key $(user_specified_key) // Create a secret configuration for IPsec, using a user-specified key
  3. kmeshctl secret get // Get the secret information in the kmesh-system namespace
  4. kmeshctl secret delete // Delete the secret in the kmesh-system namespace

Currently, all these operations are performed by default on the kmesh-ipsec secret.
@hzxuzhonghu @LiZhenCheng9527

@hzxuzhonghu
Copy link
Copy Markdown
Member

/lgtm
/approve

@kmesh-bot
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hzxuzhonghu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kmesh-bot kmesh-bot merged commit 553d7f7 into kmesh-net:main Aug 12, 2025
12 checks passed
@zrggw zrggw deleted the start_with_ipsec branch August 13, 2025 01:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved kind/bug Something isn't working kind/enhancement New feature or request lgtm size/S

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Kmesh cannot start when --enable-ipsec=true and no IPsec pre-shared keys set by kmeshctl

5 participants